Read XML from queue using MQ connector stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
svga
Participant
Posts: 73
Joined: Thu Aug 07, 2008 6:31 am
Location: Syracuse

Read XML from queue using MQ connector stage

Post by svga »

Hi -

I'm getting the below error messge while reading XML transactions from a queue. I have set datatype as longvarchar and length as 999999999.
Can anyone please let me know what could be the possible root cause for this issue?

Fatal Error: Virtual data set.; output of "MQ_Connector_ReadText": the record is too big to fit in a block; the length requested is: 521343, the max block length is: 131072.
soumya5891
Participant
Posts: 152
Joined: Mon Mar 07, 2011 6:16 am

Re: Read XML from queue using MQ connector stage

Post by soumya5891 »

Increase the size of the following two variables to a bigger value
APT_PHYSICAL_DATASET_BLOCK_SIZE,
APT_DEFAULT_TRANSPORT_BLOCK_SIZE
Soumya
svga
Participant
Posts: 73
Joined: Thu Aug 07, 2008 6:31 am
Location: Syracuse

Post by svga »

I did that. its works!! Thank you...

But I'm getting new issue. The XML records are getting written to the sequencial file. Seeing the below texts while viewing the data from seq stage. Do you have any solution for this?


<<<CC_LER>>>WebSphereMQConnector<<<CC_LER>>>5.3<<<CC_LER>>>ccwsmq<<<CC_LER>>><<<CC_LER>>><?xml version='1.0' encoding='UTF-16'?>
<Properties version='1.1'><Common>
<Context type='int'>0</Context>
<Variant type='string'></Variant>
<DescriptorVersion type='string'>1.0</DescriptorVersion>
<PartitionType type='int'>-1</PartitionType>
soumya5891
Participant
Posts: 152
Joined: Mon Mar 07, 2011 6:16 am

Post by soumya5891 »

I didn't get your point.Can you explain it briefly
Soumya
svga
Participant
Posts: 73
Joined: Thu Aug 07, 2008 6:31 am
Location: Syracuse

Post by svga »

Hi -

The job ran successfully but when I try to view the XML data on the target (Seq stage) I'm not getting the actual XML records instead getting the above data.
kandyshandy
Participant
Posts: 597
Joined: Fri Apr 29, 2005 6:19 am
Location: Singapore

Post by kandyshandy »

You need to share your input data and expected output data.
Kandy
_________________
Try and Try again…You will succeed atlast!!
kandyshandy
Participant
Posts: 597
Joined: Fri Apr 29, 2005 6:19 am
Location: Singapore

Post by kandyshandy »

You need to share your input data and expected output data.
Kandy
_________________
Try and Try again…You will succeed atlast!!
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

....you have to pass the xml content thru one of the xml stages first....xmlInput or xmlStage (you will have to learn how to use those stages)....but if you are getting the xml content in your target, whit its tags, then it sounds like you are using the MQ Connector perfectly!

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
soumya5891
Participant
Posts: 152
Joined: Mon Mar 07, 2011 6:16 am

Post by soumya5891 »

You need to parse the XML first by giving the xpath.Then only you will be able to view the XML data
Soumya
svga
Participant
Posts: 73
Joined: Thu Aug 07, 2008 6:31 am
Location: Syracuse

Post by svga »

Hi - Do I need to parse each and every tags using XML input stage and then write them to the sequencial file stage. Please advise.
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Yes. I suggest that you separate the two technologies and learn each one separately before trying to combine them. Get your MQ Connector working, sending your xml to disk. Once you are comfortable with the way you are reading the queue, start working on xml. Write and test separate jobs that read the xml from disk and parse it....you have to import the metadata and describe this to the xml stage so that it can be read correctly. Once you have that working, then you can start to combine the two. Both Stage types involve complex technology. Get comfortable with them separately and then they can easily be combined.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply