DSXchange

Posted: **Wed Jul 14, 2010 7:25 am**

Hi,

We have a wrapper script that runs a MQ job (which pulls from message queue) every five seconds. The job is kicked off fine from Autosys, when we wanted to stop the job from Autosys, it is not stopping.

Basically our requirement is to run this job between 9AM and 5PM.
Any suggestions please.

Thanks

Posted: **Wed Jul 14, 2010 8:13 am**

Why not just have the wrapper check the time and stop looping?

Posted: **Wed Jul 14, 2010 8:19 am**

That job design sounds off to me... Most of the MQ stuff I've worked on has been "always on". Instead of running the job every five seconds you start the job and it runs all day waiting for messages to process. When you need to kill it you put a special message in the queue that tells the job "time to stop".

Posted: **Wed Jul 14, 2010 8:35 am**

Hi,
The job basically sleeps for 5 seconds if the queue is empty.
We dis discuss about the 'Stop Message'. As there are a lot of front end applications sending these messages, could not implement that.
One option as per Craig is to do it through the wrapper.
I am curious if there are any other viable options.
Appreciate your response.
Thanks

asorrell wrote:That job design sounds off to me... Most of the MQ stuff I've worked on has been "always on". Instead of running the job every five seconds you start the job and it runs all day waiting for messages to process. When you need to kill it you put a special message in the queue that tells the job "time to stop".

Posted: **Wed Jul 14, 2010 8:55 am**

Your wrapper script needs to trap the signals and send a stop to your job. I posted that code before. I will post a link.

viewtopic.php?t=108490&highlight=trap

Posted: **Wed Jul 14, 2010 8:59 am**

Here is the rest of the code:

Code: Select all

function ExitProcFunction 
{ 
# -----------------------------------------------------------------
# This function will stop the DataStage job running. 
# -----------------------------------------------------------------
let ExitArgAsReturnCode=${1}
# -----------------------------------------------------------------
# If the exit code is specified, exit with that code. 
# -----------------------------------------------------------------
    if (( $# > 0 )) 
    then 
        if (( ${JobWasRunningSw} ))
        then
            if (( ${JobRunningSw} ))
            then
                print " $DSPath/dsjob -stop $ProjectName $JobName" 
                $DSPath/dsjob -stop $ProjectName $JobName
                sleep 10
            fi
            JobDetailSw=0
            # EndScriptFunction $ExitArgAsReturnCode
            EndScriptFunction 3
        else
            CatLogFunction
        fi
        exit $1
    else 
        exit 0 
    fi 
}

Posted: **Wed Jul 14, 2010 10:06 am**

Hello HondaAccord94

How about using the MQ's trigger capability? You can have MQ trigger the datastage job using DSJOB when a message arrived to the monitored queue

The DataStage job, a mutiple instance regular job, triggered by MQ read/process and delete all messages from the queue. After reading all messages the job naturally and cleanly finish, so you take Autosys out of the solution and have your datastage job executed on demand only when the queue is empty and a new message arrives

We implemented this design for a near real time solution and a high volume messages system and work like a charm

Regards

Posted: **Wed Jul 14, 2010 12:34 pm**

Ken, Thank you for the direction.

Julio, I am curious to know how I can achieve that with DSJOB? Can you please throw some more light on it. And our process runs through iterations...so it goes in to MQ gets a message out and creates an xml file, goes in gets one out creates an xml file. Using start loop and end loop activity. The reason we are doing this because we needed to tag each message with its message id and create an xml.

Thanks for your responses.

Posted: **Wed Jul 14, 2010 1:59 pm**

Honda...

We have a regular script to run DS jobs that used DSJOB command line. This script is used to execute DS code from the enterprise scheduler. We call same script, passing the job name that process the messages, from MQ automatically when a message arrives to the queue and the queue happen to be empty. If the queue is not empty the DS job is running, so MQ just add the new message to the queue...

To set up the MQ piece just google it and play with it a bit, don't hesitate to ask for more details. "Triggering application with MQ" should give you a lot of hits in a Google search

Below is a good one:
http://publib.boulder.ibm.com/infocente ... 13830_.htm

Regarding the DataStage job, we used MQ Connector stage(8.x) because it has a litany of nice properties that gives you more control. "Message quantity" property set to -1 allow to read/process all available messages in the queue, "Wait time" set to "X" to wait "X" number of seconds after processing the last available message before shutting down the job. After reading the message, you control the flow as you normally would do.

I guess you don't need to do the looping that you mentioned in your post If you parse the messages one at a time from the queue. This is similar to any other source in DataStage

have fun!

Posted: **Wed Jul 14, 2010 2:52 pm**

Who is Ken?

Posted: **Wed Jul 14, 2010 2:54 pm**

Thanks a lot, Julio.
That is some great information.

Posted: **Wed Jul 14, 2010 6:00 pm**

kduke wrote:Who is Ken?

Why, you are of course.

Posted: **Thu Jul 15, 2010 9:55 am**

I will call you Ray from now on.

Posted: **Thu Jul 15, 2010 9:56 am**

As long as you don't make me tell bad jokes too, I'm cool with that. :D

Posted: **Thu Jul 15, 2010 10:06 am**

Where is my Louisville Slugger?

DSXchange

Autosys to stop DS job

Autosys to stop DS job