Issue in Reading a Sequential File

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mac4rfree85
Participant
Posts: 126
Joined: Thu Jul 01, 2010 11:39 pm

Issue in Reading a Sequential File

Post by mac4rfree85 »

Hi Guys,

I am trying to read XML file using the Sequential file stage. I am giving a Record delimiter as </Transaction> (closing Transaction Tag).
But for some reasons, after 10 records the job is throwing error that delimiter is not present. But i am able to see the delimiter. In fact, i did a grep on the Closing tag to check the number of records which was exact number of records which is needed.

I created a new XML from my source MQ queue. Even the new XML file, aborted after processing 10 records(using peek stage am able to say this). I even removed the 11th record and kept the remaining records. Still, i am having the same issue.

So, can somebody let me know what is going wrong.

Cheers!!
Mac4rfree
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Issue in Reading a Sequential File

Post by chulett »

mac4rfree85 wrote:I am trying to read XML file using the Sequential file stage.
Why? An XML file is not a "sequential" file in the sense that the Sequential File stage understands reliably. Technically they are one long stream of characters with no individual records. Why not use the stage that was meant to read XML, the XML Input stage?
-craig

"You can never have too many knives" -- Logan Nine Fingers
mac4rfree85
Participant
Posts: 126
Joined: Thu Jul 01, 2010 11:39 pm

Re: Issue in Reading a Sequential File

Post by mac4rfree85 »

chulett wrote:
mac4rfree85 wrote:I am trying to read XML file using the Sequential file stage.
Why? An XML file is not a "sequential" file in the sense that the Sequential File stage understands reliably. Technically they are one long stream of characters with no individual records. Why not use the stage that was meant to read XML, the XML Input stage?
I agree to it. But this how the client needs it. As there are already existing jobs which is designed in this manner. I added an environmental variable which helped me to resolve the issue. "$APT_DELIMITED_READ_SIZE = 990000000" This is what i added. Now i am able to read the file. :)
Mac4rfree
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

<rant>Resist stupid requirements!</rant>
Please post your client's reasons for "needing" it.
Does your client have pointy hair?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Nobody "needs" it that way. :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply