Hello,
I have question related to DataStage MQ plug-in.
I know its queue, but want to clear my doubt on how order of message on MQ preserve when messages its read via DataStage.
We have datastage jobs reading xml data from MQ, perform set of xml parsing and load data to target tables. We expect volume of around 70000 xml messages per set. And each xml message having many to many relationship.
We are planning to read and process 10000 xml messages (without destrctive read) in a single job run and use same job to run in loop to fetch all messages at the end. I want to know will it always read first 10000 xml messages?..will it preserve the order on MQ?
I have another job at the end which performs destructive read. If I defined the same limit 10000 xml message in the job which perform destructive read, will it always delete first 10000 xml message..meaning will it maintain the order? or it will randomly delete the messages?
MQ read; order of xml messages on queue maintain.
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 62
- Joined: Tue Sep 21, 2004 10:24 am
- Location: IBM - Chicago Area
MQ Queues are first-in-first-out, so the ordering will remain in place - unless you change it explicitly in your DataStage job. You can confirm this by reading and checking the internal 24 byte "MsgId" unique number - these will be incremented by 1 at the rightmost position for each record.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
...just be sure that you aren't using any of the various filtering capabilities of the Stage...or if you are, use them carefully and do a lot of testing.....that could change things, or at least skip messages along the way.
Ernie
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
-
- Premium Member
- Posts: 62
- Joined: Tue Sep 21, 2004 10:24 am
- Location: IBM - Chicago Area
Thank you, its helped..ArndW wrote:MQ Queues are first-in-first-out, so the ordering will remain in place - unless you change it explicitly in your DataStage job. You can confirm this by reading and checking the internal 24 byte "MsgId" unique number - these will be incremented by 1 at the rightmost position for each record.
Regards,
Gaurav