Page 1 of 1

controlled writing to MQ in Datastage

Posted: Fri Jun 18, 2010 2:32 am
by dsguyok
In Datastage 8 we need to write to a MQ queue which has a limited depth (of course). We have more messages to put on there than space available on the queue (ie depth).

Instead of increasing the queue depth, we thought about "sleeping" the queue-writing job until more queue slots are available ie to avoid overloading the queue.

Is this a good way about doing it in Datastage and if so how would we make a job sleep and check the queue depth periodically?

Thanks

Posted: Fri Jun 18, 2010 5:04 am
by ArndW
We had a similar requirement in the last project and, after much deliberation, it was decided that the only safe method was to increase the queue depth and ensure that the queue reading process worked faster. Putting a sleep or intentional slowdown in DataStage jobs is not usually a good idea. If you cannot increase the queue depth, I'd split the processing into two stages - one DataStage process that writes to a sequential file, then another process that takes the sequential file and moves it to MQ. This process could be a simple server job to read the file, do some SLEEP() logic in a transform stages, then write to MQ.