MQ Stage Message read

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Amit Jaiswal
Premium Member
Premium Member
Posts: 38
Joined: Fri Apr 22, 2005 6:07 am

MQ Stage Message read

Post by Amit Jaiswal »

Hi All,

I have designed a job to read messages as job names from the MQ contineously (24x7 without stop). I have used wrapper stage to execute a shell script by passing job name as parameter. I want whenever MQ Stage reads the message it should send it to the output so that Wrapper stage triggers the shell script row by row, so that my jobs get triggered. However, I found that though MQ is reading messages but its not sending it to output. Messages are only going to output if job is getting finished. This I have verified by setting some wait time.
However, I want to run my job without stop and still need messages in the output. Please let me know if I am missing any setting.

Thanks in advance.
-Amit
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

You need to come up with a "pick n throw" design. This can be achieved by making the message reading job to read (pick) one message at a time and come out and trigger (throw) the subsequent jobs it needs to call and return the control back to the message reading job to wait/pick the next message without waiting for the result of the subsequent jobs(making them run independently).
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

I can only guess on this one, but it sounds like it might be an end-of-wave issue. It's very similar to what would happen if you were to use the XMLOutput Stage, or the aggregator...stages that need "end of set/job" to complete as when running in batch. The MQ Stage leaves you with an "always on" job, and so there is no "end of set" indicator (at least not in 7.x --- in v8 the MQ Stage has a check box to force "end of wave" thus simulating end of job).

If the wrapper you are writing needs the end of wave, you might try using the BASIC transformer instead and using the sdk function called DSUtilityRunJob to invoke your "other" jobs. end-of-wave might not even be your problem, but this is worth a try.

Ernie
Amit Jaiswal
Premium Member
Premium Member
Posts: 38
Joined: Fri Apr 22, 2005 6:07 am

Post by Amit Jaiswal »

Hi Erne,
Its working with DSUtilityRunJob routine. However, triggering the jobs in sequence. I want it should not wait to finish one job before starting another job.
Thanks,
-Amit
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

...been many years since I've looked at the guts of that routine, but go into your Routines/sdk category of the manager, and look at the guts of it. I suspect that you can cut/paste the contents of it into another Routine and hack out the "-wait" option that is in there somewhere. This routine, as I recall, uses the UV/Basic implementation of the job control API, which ought to support the wait as well as the nowait options. See chapter 19 of your servjdev.pdf ...

Good luck. Sorry I don't have time to look and hack it up for you. It sounds like a fun one.

Ernie
Post Reply