Reading XML file using MQ Stage in Datastage 9.1

[nyk1989]

Hi,

I am trying to read a XML file using MQ stage in DS.
Design - MQ -> XML Stage -> Seq File
But its throwing me the following error-

XML_1,0: 2015-12-14 09:43:40,801 Error [XML_Parser] [] Unable to parse the XML stream: <?xml version='1.0' encoding='UTF-8'?>
2015-12-14 09:43:40,821 Error [XML_Parser] [] com.ibm.e2.xml.exceptions.XMLRuntimeException: CDIER0007E: An error occurred while reading input: language=XML, cause=javax.xml.stream.XMLStreamException: The root element is required in a well-formed document., systemId=null, line number=2, column number=1, step=XML_Parser, source=<?xml version='1.0' encoding='UTF-8'?>.
XMLRuntimeException$FactoryImpl.errorWhenReadingXML() 13
RuntimeNodeBase.consumeEvent() 752
RuntimeNodeBase.consumeEventWithLogging() 2318
RuntimeNodeBase.consumeAction() 482
RuntimeNodeBase.execute() 339
NodeExecutor.processOneEvent() 163
XMLReaderUtils.runExecutorStreaming() 201
XMLInputStreamingHandler.itemEnd() 44
AbstractTraverser.handleEventForHandler() 663
AbstractTraverser.handleEvent() 534
ItemTraverser.handleCurrentState() 91
AbstractTraverser.continueTraversal() 475
AbstractTraverser.startTraversal() 463
ItemTraverser.itemBegin() 122
AbstractTraverser.handleEventForHandler() 668
AbstractTraverser.handleEvent() 534
VectorTraverser.handleCurrentState() 133
AbstractTraverser.continueTraversal() 475
AbstractTraverser.startTraversal() 463
AbstractRuntimeFrame.runTraverser() 1426
AbstractRuntimeFrame.runTraverser() 1409
UserRuntimeOperatorAdapter.runTraverser() 138
AbstractRuntimeOperator.runTraverser() 199
AbstractXMLProviderRuntime.process() 254
UserRuntimeOperatorFrame.process() 93
OperatorController.callOperatorProcess() 314
OperatorController.runOperator() 264
OperatorController.doReadyToExecute() 168
OperatorController.runDataStateTransistion() 123
OperatorController.runTransitions() 85
OperatorController.runOperatorStep() 63
OperatorTask.runFrame() 86
OperatorTask.execute() 40
AbstractTask.run() 27
ThreadPoolExecutor$Worker.runTask() 886
ThreadPoolExecutor$Worker.run() 908
Thread.run() 736


XML_1,0: Fatal Error: CDIER0007E: An error occurred while reading input: language=XML, cause=javax.xml.stream.XMLStreamException: The root element is required in a well-formed document., systemId=, line number=2, column number=1, step=XML_Parser, source=<?xml version='1.0' encoding='UTF-8'?>.

Any idea what is this and how can we resolve this?

I tried to read the same XML file using External Source Stage and the file got parsed properly and data got loaded into the Seq file.
So, it means that the XML file is correct and it conforms with the XSD.

But its not working with the MQ.. Any suggestions??

[chulett]

I tried to read the same XML file using External Source Stage and the file got parsed properly and data got loaded into the Seq file.
So, it means that the XML file is correct and it conforms with the XSD

Actually no, no it doesn't.

Post your XML file or at least a representative snippet from its head.

[JRodriguez]

Hi nyk1989,

How about dumping the XML file from MQ to a file first? Test that the file in the queue is still good....

Regards

[nyk1989]

This is a snippet of the XML file -

Code: Select all

<?xml version='1.0' encoding='UTF-8'?>
<PARENT_LEVEL_EVENT_DATA timeOfOrigin="23-NOV-15 06.06.06.000000000">
    <INSTITUTION>
        <INSTITUTION_ID>1</INSTITUTION_ID>
        <INSTITUTION_NAME><![CDATA[ANZ Bank]]></INSTITUTION_NAME>
        <BATCHNAME_PATTERN><![CDATA[PCO %P %A %F %U %D %T]]></BATCHNAME_PATTERN>
        <EMAIL_CONTACT><![CDATA[]]></EMAIL_CONTACT>
        <EMAIL_TRADE_REP><![CDATA[]]></EMAIL_TRADE_REP>
        <EMAIL_ROLL_REP><![CDATA[]]></EMAIL_ROLL_REP>
        <EMAIL_M2M_REP><![CDATA[]]></EMAIL_M2M_REP>
        <EMAIL_NAV_REP><![CDATA[]]></EMAIL_NAV_REP>
        <CONTACT_INFO><![CDATA[]]></CONTACT_INFO>
        <COMMENTS_TEXT><![CDATA[]]></COMMENTS_TEXT>
        <LAST_UPDATED>12-OCT-12 11.19.18.000000000</LAST_UPDATED>
        <LAST_UPDATED_BY><![CDATA[J006165]]></LAST_UPDATED_BY>
        <VERSION_NUMBER>3</VERSION_NUMBER>
    </INSTITUTION>

[nyk1989]

How about dumping the XML file from MQ to a file first? Test that the file in the queue is still good....

I am able to successfully dump the XML file from MQ into a Seq file. And the XML file matches with the file coming from the source.

[nyk1989]

I also tried the job with a sample XML file as below -

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<note>
<to>Tove</to>
<from>Jani</from>
<heading>Reminder</heading>
<body>Don't forget me this weekend!</body>
</note>

But the job aborts with the same above mentioned error for this XML file also.

Process followed -

1) Loading MQ with the XML file
2) Defined only 1 output column containing this XML from MQ stage in DS
3) In XML Assembly, using 'String Set' option
4) Providing the XSD file in Document Root
5) Strict Validation option selected
6) MApping the source to the target

Please let me know if anymore details required.. Any idea what's the issue here??

[nyk1989]

Guys,

Any suggestions on how to resolve this?? Thanks!!

[eostic]

Yes. Separate ENTIRELY the two processes during development. Make two copies of your current Job. In one, just target a Sequential Stage. Send it the one single big column. Make it a longvarchar, and some long length like 99999. Give it a format of NONE for delimiter and quote character. Send it to /tmp on the DataStage Engine. Then try to open it, using a browser on that platform. See if it opens as normal xml successfully. If it does, it is probably clean, but could still have encoding issues. If it fails, then you know something is wrong before you even get into DS.

Now, assuming it does open successfully in a browser local to that platform.....then have a Job that reads that flat file using the Hierarchical Stage. Does it get the same error?

If so, then there is something wrong with the header of the file, or that DS "thinks" is wrong with the header. Encoding issues are often the problem.....but that's just a guess. Debug it in pieces with "the" exact file that is created from the queue. Put the Job pieces back together only once you have them working independently.

Ernie

[nyk1989]

Yes. Separate ENTIRELY the two processes during development. Make two copies of your current Job. In one, just target a Sequential Stage. Send it the one single big column. Make it a longvarch ...

I was not able to read this solution as I am not a Platinum member.

Still, I tried to debug the job and now I am able to parse my XML file when the entire document is coming out as a single message. (Soln:- I made the message to contain the absolute path of the xml file).

Hence, my question now is - Is it mandatory for the XML stage to receive the entire XML document in a single message or the XML document can be divided into several messages?? And is my above soln. correct or is there any alternative way to also do that?

Thanks!!