Messages are deleting from Queue if job aborts

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
padmavathi
Participant
Posts: 8
Joined: Tue Sep 01, 2009 6:16 am

Messages are deleting from Queue if job aborts

Post by padmavathi »

Hello All,
I have the requirement to read xml messages from Queue and apply transformations like mandatory checks, rejection and load data to database.

I have selected the Delete (Under Transaction) option in MQ read stage. The messages are deleted from queue if job aborted in intermediate stages like transform stage.

Please suggest is there any option to keep messages in queue till my job completes successfully. The messages has to get delete once job compeltes successfully.

Thanks
Padma
v2kmadhav
Premium Member
Premium Member
Posts: 78
Joined: Fri May 26, 2006 7:31 am
Location: London

Post by v2kmadhav »

Can you not run your job in "Keep" mode and then flush the queue once the jobs runs finishes ?
padmavathi
Participant
Posts: 8
Joined: Tue Sep 01, 2009 6:16 am

Post by padmavathi »

To flush the queue, do we require another job? Flush means reading Queue again with Delete option or any other option available? Can't we achive the same in single job
v2kmadhav
Premium Member
Premium Member
Posts: 78
Joined: Fri May 26, 2006 7:31 am
Location: London

Post by v2kmadhav »

If your target database is compatible with a DTS stage (and ofcourse if you have it and if your release support it) you could use that - and messages are handled within the job the way you want it without any additional logic.

But in the end i guess it comes down to what the business want you to do in such a scenario.. I think unless you introduce a mechanism to capture rows and retry when needed, you cannot use any of the other options..

Yes flush queues - another job with delete mode, or perhaps a script thats called in ur after job routine.
mobashshar
Participant
Posts: 91
Joined: Wed Apr 20, 2005 7:59 pm
Location: U.S.

Post by mobashshar »

Why not move the data to WorkQ in source? It is required by DTS. Also you can choose the option to use a RejectQ, so the data will be cleared from WorkQ and written to rejectQ
soumya5891
Participant
Posts: 152
Joined: Mon Mar 07, 2011 6:16 am

Post by soumya5891 »

Extract from queue and stored the xml into a single field in a dataset.Perform this into an single job. You can read the message in delete mode.

Then in next job you can use the dataset for xml parsing and transformation. If any abortion occur in this job then xml will be as it is in the dataset.
Soumya
Post Reply