MQ; queue depth check via DS job

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Gaurav.Dave
Premium Member
Premium Member
Posts: 62
Joined: Tue Sep 21, 2004 10:24 am
Location: IBM - Chicago Area

MQ; queue depth check via DS job

Post by Gaurav.Dave »

Hi

We are using DataStage 7.5 on AIX 5.1

In our job we are pulling data from MQ stage. We are expecting about 100K messages by year end....

We r working on loop approach to read 5000 messages at a time, perform xml parsing and load to target and after that running 2nd job to perform destructive read with 5000 messages. So, we can occupy less memory.

Wondering is it anyway we can check qdepth() and control the loop to terminate when there are no messages into MQ??

Thanks..
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

No need, really.... just use the timeout capability. The job will terminate nicely when there are no messages if you set a really short timeout period.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Gaurav.Dave
Premium Member
Premium Member
Posts: 62
Joined: Tue Sep 21, 2004 10:24 am
Location: IBM - Chicago Area

Post by Gaurav.Dave »

eostic wrote:No need, really.... just use the timeout capability. The job will terminate nicely when there are no messages if you set a really short timeout period.

Ernie
Thanks Ernie, Thanks for your reply on IBM techlist too...

Yes, if we supply timeout period as "0" then job terminate. But what I am looking for is that I want to break the loop by checking queue depth empty.
Because, messages on queue will vary starting from few thousands in the first month and by year end it will rise to 100k. And our process is weekly process. So, we have written job taking care of 5000 messages at a time right now to avoid memory issue and we have create loop using sequencer job to repeat it 20 times. so total will be 100k.

But when we have initally few thousand records its waste of time to running loop 20 times. Thats the reason i m checking possiblity if we can somehow grep queue depth parameters and break the loop when queue is empty....

Regards,
Gaurav
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

You can't do it from the MQ stage. If you have been granted the MQ rights, you can execute the appropriate "runmqsc" UNIX command sequence to return the queue depth to your calling sequence and branch accordingly.
Post Reply