Read XML from MQ

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
naveend
Participant
Posts: 11
Joined: Mon May 12, 2008 12:49 pm

Read XML from MQ

Post by naveend »

I am trying to develop a job to read XML from MQ. What I want to do is read the message from the Q and parse the header of the XML and validate the type of transaction it is, and based on the value, pass it to an XML output stage and write each XML to a separate file. When I move the message to the XML stage, I am not getting any output. I dumped all the messages into a file and when I try to process the XML, only the first XML is getting processed. I know there is not problem with the XML as I am able to process the individual XMLs. But when they are coming thru the MQ, I am not able to process. I am getting different types of transactional data and the header is the same on all the XMLs. I am trying to validate the "<ClientTransactionName>" to create different files. Can someone help me out?

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE TCRMService SYSTEM "tCRMResponse.dtd">

<TCRMService>

<ResponseControl>

<ResultCode>SUCCESS</ResultCode>

<ServiceTime>214</ServiceTime>

<DWLControl>

<clientTransactionName>SAP Company Hierarchy</clientTransactionName>

<requesterLanguage>100</requesterLanguage>

<requesterLocale>en</requesterLocale>

<requesterName>cusadmin</requesterName>

<requestID>100625</requestID>

</DWLControl>

</ResponseControl>

<TxResponse>

<RequestType>CATGetAllHierarchyAncestorsNodes</RequestType>

<TxResult>

<ResultCode>SUCCESS</ResultCode>

</TxResult>


View user's profile Send private message
throbinson
Charter Member
Charter Member
Posts: 299
Joined: Wed Nov 13, 2002 5:38 pm
Location: USA

Post by throbinson »

XMLOutput Stage? I would have thought the XMLInput stage is more appropriate. Is the entire XML in a single MQ Message body? If so, it should be read from MQ that way and passed en-masse to the XMInput Stage as a longvarchar chunk. Parse,filter,pass and kick the whole XML into the donwstream links from the XMLInput stage any way you want and go from there.
naveend
Participant
Posts: 11
Joined: Mon May 12, 2008 12:49 pm

Post by naveend »

I am using the XML Input after the MQ. Here is how my job is:

MQ==>XML INPUT==>XFM==>XML OUTPUT (multiple)
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

This should be no problem...but check a variety of things carefully:

a) that you are getting the WHOLE document in your "message" column (you really should have only one column on the output link of the MQ stage) --- be sure nothing is getting truncated.

b) "(a)" assumes that one message = one whole document.

c) pass the original message column thru the xmlInput Stage. You'll be parsing out your header field as apprpriate with the necessary XPath (such as /TRCM..../ResponseControl.../.../.../text() ) but also carry the incoming message on both your input and output link, with nothing in the Description property.

d) got to a transformer where you make your decision on which file to go to..... and have a separate link for each possible output file that you are branching on.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

...and btw.... no need in this case to ever use XMLOutput. Just use Sequential Stages to write out the file, with no delimiter and no quote characters.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
naveend
Participant
Posts: 11
Joined: Mon May 12, 2008 12:49 pm

Post by naveend »

I am getting the whole message in one column. I created another job to move the message to the XML input stage and put the result into a SEQ file.

MQ==>XMLInput==>SEQ

I coded XPATH for the clientTransactionName and am not getting any output from the XML stage.

It looks like the XML stage is not recognizing the XML at all. I have the 'repetition element required' checked on the stage. I have the xml source column set as "XML document". I hope this is correct.
John Smith
Charter Member
Charter Member
Posts: 193
Joined: Tue Sep 05, 2006 8:01 pm
Location: Australia

Post by John Smith »

Hi ,

You need to make sure that you are getting the entire message (XML file) from MQ. So before you run it into the XML stage, just create a simple job like this : MQ --> Seq file

If the Seq file is created as a good XML file then you're good to go e.g
Seq File --> XML Input

You need to debug where your problem is happening.

Have fun!

Rgds,
JS
naveend
Participant
Posts: 11
Joined: Mon May 12, 2008 12:49 pm

Post by naveend »

I had been able to dump the message from the MQ into a SEQ file and then process it thru the XML input stage. The issue is, when I feed the message from MQ into the XML stage, I am not getting any output. I checked the DTD and the xpaths and it looks good. I would appreciate any new suggestions.
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Something is different when you read it in from MQ...you are getting too much....or too little, or there are garbage characters, or it's a different Job and there are case specific issues in your xpath....etc..... Clearly the text coming from MQ is not the same as what you are reading from your flat file......... or else maybe you are using different links. Save your flat file XML job as something else and delete the sequential file stuff and replace it with MQ........ not sure what else to suggest...something is clearly different --- either in the data, or in your Job.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply