Stopping an MQ-reading job

hobocamp · Post by **hobocamp** » Tue Mar 03, 2009 3:37 pm

In our environment one of our DS jobs basically runs 24 hours a day. It reads XML messages from MQ, parses them and loads the data into an Oracle table.

I've now had a request to schedule the job so that it starts at a certain time of the day, and ends at a certain time of the day. But so far I haven't read anything that indicates there's a graceful way to stop this type of job.

My first thought was to make it a Sequence job, which would accomplish the scheduling start-up requirement. But is there a way within a sequence job to command the job to stop after, say, running 10 hours? Would it be possible with perhaps with something like the Sequence Terminator Activity?
Thanks for any advice.

Tom

ray.wurlod · Post by **ray.wurlod** » Tue Mar 03, 2009 5:07 pm

I normally use a file. Check periodically that the file exists (or does not exist) using a WaitForFile activity, and rename (or create) the file to notify the job sequence to shut down.

Kryt0n · Post by **Kryt0n** » Tue Mar 03, 2009 5:36 pm

Haven't had the fortune of using MQ and DS together yet... came close but offshoring was closer...

Have you tried looking in to the "End of data message type"? Reading the help files implies it is fairly straight forward, just throw one of these message types at the queue when you want shutdown...

Use this property to specify the message type in the message header that specifies the end of the message reading process.

The connector stops reading messages when it receives a message that matches the type that is specified in this property. The exception to this is when there are additional messages in a group of messages and the message group must be assembled. In the group assembly scenario, the rest of the messages in the group are also read.

You can specify an integer between 0 and 999999999.

There is no default value.

hobocamp · Post by **hobocamp** » Tue Mar 03, 2009 9:38 pm

Thanks guys for the replies.

Kryt0n - If I'm understanding correctly, I'm not sure that solution would work for me, as there is no real end to the stream of messages. They can come in in any order, throughout the day, or not at all.

Ray - I think what you're suggesting would work for me. I'll give this a try.

Thanks again.

Tom

Kryt0n · Post by **Kryt0n** » Tue Mar 03, 2009 10:14 pm

You don't need the end of message from source, it can be initiated from anywhere as all it needs to do is add a message with relevant message type to the queue.

Theoretically, you can create a job that posts the message to the queue, this job can then be kicked off via scheduler or ad-hoc to bring the MQ job down.

I have no MQ to play with so sits as a nice idea in my head...

hobocamp · Post by **hobocamp** » Thu Mar 05, 2009 1:56 pm

Ray -

Building on your suggestion, would it make sense then to have the Wait For File stage flow to the Execute Command stage, which would execute the DSStopJob command?

Thanks.

Tom

eostic · Post by **eostic** » Thu Mar 05, 2009 3:10 pm

The most elegant solution for stopping the job is either the MQ message type (pick a big integer value like 998999), or to have a long timeout period (say, a few hours). In those cases it will come down cleanly. Anything else will likely be an aborted Job, potentially causing other problems.

Ernie

ray.wurlod · Post by **ray.wurlod** » Thu Mar 05, 2009 3:26 pm

DSStopJob is not a command. But you're on the right track. A Terminator activity would do it.

throbinson · Post by **throbinson** » Fri Mar 06, 2009 4:33 am

I've implemented the End of Message technique and it has worked without incident for 6 years. It would require two jobs; one to put the stop message onto the queue and one job to take it off. It is elegant and simple and for those reasons also robust.

IBM Analytics Champion 2009 - 2020 · Post by **asorrell** » Fri Mar 06, 2009 9:49 am

I have to agree - I always used a "stop" message with all "nines" as the best way to shut down an MQ job. Just created a job to slap one on the queue. I didn't need a job to take it off because I had the main job remove it from the queue as "processed" before shutting down cleanly.

Can we consider this thread "resolved"? Or do you need further assistance?

hobocamp · Post by **hobocamp** » Tue Mar 10, 2009 7:04 am

I'll consider it resolved, and mark the thread as such. Thanks everyone for the information and suggestions.

Tom

stpat389 · Post by **stpat389** » Tue Mar 17, 2009 10:59 am

What does "processed" mean? I have implemented the End of Day message and it successuly stops the stop. However, it leaves the End of Day message on the queue. What do I need to do to get the message to be "processed" and deleted off the queue when the job is done.

throbinson · Post by **throbinson** » Wed Mar 18, 2009 6:16 am

I took asorrell's use of "processed" to mean he sent the stop message MsgId down a link that deleted the message via the "old" process of two writes to the queue using sync-point control. Version 8.x might have a fancier way of doing it. The other method would be to do a destructive read from the queue of the "stop" message in another job that only runs after the real job has stopped. You would filter on MsgType='99999' or whatever Msgtype you used to identify the "stop" message.