How to Handle the MQ Failure

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
pksahu
Participant
Posts: 29
Joined: Wed Jul 25, 2007 5:07 am

How to Handle the MQ Failure

Post by pksahu »

Hi,
I want to Handle the MQ Connector Stage sequence while the sequence abort in the middle (while reading the Message from the Queue).

Example: Suppose I have 100 messages in the queue and writing into a Dataset. If due to some reason the message queue failed after reading 50 messages, how I should handle to read the entire 100 message again so that I will get the full set of data.

Can anyone give some idea what should I follow to archive this?

Regards,
Pksahu
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Well.... you probably don't want to choose a destructive read in this case. A DataSet is just a file on disk --- there is no ensured delivery for that. To be safe, "browse" your messages and don't remove them from the original queue until you are sure that things have been completed successfully on the other side.

An alternative is to look at what the Connector can do for "work queues"....put the messages into a work queue (which the Connector is able to do under "syncpoint"...meaning that they get moved with assured delivery in an MQ transaction and won't be dropped) and then delete them later from there when you have things correctly in your target, whatever that target might be.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply