Autosys to stop DS job
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 46
- Joined: Tue Aug 10, 2004 11:07 am
- Location: Mclean VA
Autosys to stop DS job
Hi,
We have a wrapper script that runs a MQ job (which pulls from message queue) every five seconds. The job is kicked off fine from Autosys, when we wanted to stop the job from Autosys, it is not stopping.
Basically our requirement is to run this job between 9AM and 5PM.
Any suggestions please.
Thanks
We have a wrapper script that runs a MQ job (which pulls from message queue) every five seconds. The job is kicked off fine from Autosys, when we wanted to stop the job from Autosys, it is not stopping.
Basically our requirement is to run this job between 9AM and 5PM.
Any suggestions please.
Thanks
That job design sounds off to me... Most of the MQ stuff I've worked on has been "always on". Instead of running the job every five seconds you start the job and it runs all day waiting for messages to process. When you need to kill it you put a special message in the queue that tells the job "time to stop".
-
- Participant
- Posts: 46
- Joined: Tue Aug 10, 2004 11:07 am
- Location: Mclean VA
Hi,
The job basically sleeps for 5 seconds if the queue is empty.
We dis discuss about the 'Stop Message'. As there are a lot of front end applications sending these messages, could not implement that.
One option as per Craig is to do it through the wrapper.
I am curious if there are any other viable options.
Appreciate your response.
Thanks
The job basically sleeps for 5 seconds if the queue is empty.
We dis discuss about the 'Stop Message'. As there are a lot of front end applications sending these messages, could not implement that.
One option as per Craig is to do it through the wrapper.
I am curious if there are any other viable options.
Appreciate your response.
Thanks
asorrell wrote:That job design sounds off to me... Most of the MQ stuff I've worked on has been "always on". Instead of running the job every five seconds you start the job and it runs all day waiting for messages to process. When you need to kill it you put a special message in the queue that tells the job "time to stop".
Your wrapper script needs to trap the signals and send a stop to your job. I posted that code before. I will post a link.
viewtopic.php?t=108490&highlight=trap
viewtopic.php?t=108490&highlight=trap
Mamu Kim
Here is the rest of the code:
Code: Select all
function ExitProcFunction
{
# -----------------------------------------------------------------
# This function will stop the DataStage job running.
# -----------------------------------------------------------------
let ExitArgAsReturnCode=${1}
# -----------------------------------------------------------------
# If the exit code is specified, exit with that code.
# -----------------------------------------------------------------
if (( $# > 0 ))
then
if (( ${JobWasRunningSw} ))
then
if (( ${JobRunningSw} ))
then
print " $DSPath/dsjob -stop $ProjectName $JobName"
$DSPath/dsjob -stop $ProjectName $JobName
sleep 10
fi
JobDetailSw=0
# EndScriptFunction $ExitArgAsReturnCode
EndScriptFunction 3
else
CatLogFunction
fi
exit $1
else
exit 0
fi
}
Mamu Kim
-
- Premium Member
- Posts: 425
- Joined: Sat Nov 19, 2005 9:26 am
- Location: New York City
- Contact:
Hello HondaAccord94
How about using the MQ's trigger capability? You can have MQ trigger the datastage job using DSJOB when a message arrived to the monitored queue
The DataStage job, a mutiple instance regular job, triggered by MQ read/process and delete all messages from the queue. After reading all messages the job naturally and cleanly finish, so you take Autosys out of the solution and have your datastage job executed on demand only when the queue is empty and a new message arrives
We implemented this design for a near real time solution and a high volume messages system and work like a charm
Regards
How about using the MQ's trigger capability? You can have MQ trigger the datastage job using DSJOB when a message arrived to the monitored queue
The DataStage job, a mutiple instance regular job, triggered by MQ read/process and delete all messages from the queue. After reading all messages the job naturally and cleanly finish, so you take Autosys out of the solution and have your datastage job executed on demand only when the queue is empty and a new message arrives
We implemented this design for a near real time solution and a high volume messages system and work like a charm
Regards
Julio Rodriguez
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
-
- Participant
- Posts: 46
- Joined: Tue Aug 10, 2004 11:07 am
- Location: Mclean VA
Ken, Thank you for the direction.
Julio, I am curious to know how I can achieve that with DSJOB? Can you please throw some more light on it. And our process runs through iterations...so it goes in to MQ gets a message out and creates an xml file, goes in gets one out creates an xml file. Using start loop and end loop activity. The reason we are doing this because we needed to tag each message with its message id and create an xml.
Thanks for your responses.
Julio, I am curious to know how I can achieve that with DSJOB? Can you please throw some more light on it. And our process runs through iterations...so it goes in to MQ gets a message out and creates an xml file, goes in gets one out creates an xml file. Using start loop and end loop activity. The reason we are doing this because we needed to tag each message with its message id and create an xml.
Thanks for your responses.
-
- Premium Member
- Posts: 425
- Joined: Sat Nov 19, 2005 9:26 am
- Location: New York City
- Contact:
Honda...
We have a regular script to run DS jobs that used DSJOB command line. This script is used to execute DS code from the enterprise scheduler. We call same script, passing the job name that process the messages, from MQ automatically when a message arrives to the queue and the queue happen to be empty. If the queue is not empty the DS job is running, so MQ just add the new message to the queue...
To set up the MQ piece just google it and play with it a bit, don't hesitate to ask for more details. "Triggering application with MQ" should give you a lot of hits in a Google search
Below is a good one:
http://publib.boulder.ibm.com/infocente ... 13830_.htm
Regarding the DataStage job, we used MQ Connector stage(8.x) because it has a litany of nice properties that gives you more control. "Message quantity" property set to -1 allow to read/process all available messages in the queue, "Wait time" set to "X" to wait "X" number of seconds after processing the last available message before shutting down the job. After reading the message, you control the flow as you normally would do.
I guess you don't need to do the looping that you mentioned in your post If you parse the messages one at a time from the queue. This is similar to any other source in DataStage
have fun!
We have a regular script to run DS jobs that used DSJOB command line. This script is used to execute DS code from the enterprise scheduler. We call same script, passing the job name that process the messages, from MQ automatically when a message arrives to the queue and the queue happen to be empty. If the queue is not empty the DS job is running, so MQ just add the new message to the queue...
To set up the MQ piece just google it and play with it a bit, don't hesitate to ask for more details. "Triggering application with MQ" should give you a lot of hits in a Google search
Below is a good one:
http://publib.boulder.ibm.com/infocente ... 13830_.htm
Regarding the DataStage job, we used MQ Connector stage(8.x) because it has a litany of nice properties that gives you more control. "Message quantity" property set to -1 allow to read/process all available messages in the queue, "Wait time" set to "X" to wait "X" number of seconds after processing the last available message before shutting down the job. After reading the message, you control the flow as you normally would do.
I guess you don't need to do the looping that you mentioned in your post If you parse the messages one at a time from the queue. This is similar to any other source in DataStage
have fun!
Julio Rodriguez
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
-
- Participant
- Posts: 46
- Joined: Tue Aug 10, 2004 11:07 am
- Location: Mclean VA