DSXchange

Fatal error:Consumed more than 100,000 bytes looking for record delimiter; aborting.

When I am trying to extract a XML with multiple files from the seqfile stage and split the data into two xml files with the business req.

my design :

seqfile ----> XML input ------>Xtans---------->Xmloutput(creating an xml file with the schema given to me)

Could you please let me know when i view the data i can see in the single
line is that right or how should i extract rom FF.

What will be the delimeter and quote for XML file and where I can identify

The problem is in the Sequential File stage; DataStage has scanned 100KB without finding the record delimiter you have specified on the format tab, and so has given up. If the source file is unterminated, then it must be fixed width, which is unlikely for XML. Therefore, I can only suggest that you have specified an incorrect line terminator (record delimiter) on the Format properties, or that you need to get a new source file, one that does have line terminators.

Search the forum for other approaches to this issue.

Hi Ray,

I work with Shilpa, and have found that the problem here is as you say. The file is not formatted in the "usual" XML format, but in fact consists of a single line. Structurally, the XML is intact and properly formatted in terms of tags and so forth.

We've found that the same file can be successfully processed in the Server canvas using the folder stage.

Is this just a limitation of the sequential file stage in the PX canvas? Or maybe there's a parameter or setting that I don't know about which will ignore or increase the max bytes read before delimiter?

We've also thought about using some awk or tr to put in newlines between tags, but thought that there might be another way.

thanks,
rob

UPDATE: I've found this post in the forums. I'll try setting the APT parameters (APT_MAX_DELIMITED_READ_SIZE, APT_DELIMITED_READ_SIZE

viewtopic.php?t=105862&highlight=sequen ... +delimiter

Is this a server job (as marked) or a parallel job (as posted)? The APT... environment variables will have no effect whatsoever on a server job.

Have you tried using 000 as the field delimiter and as the record delimiter?

ray.wurlod wrote:Is this a server job (as marked) or a parallel job (as posted)? The APT... environment variables will have no effect whatsoever on a server job.

Have you tried using 000 as the field delimiter and as the record delimiter?

We initially tried to do this with parallel job (as correctly posted). However, as part of troubleshooting and necessity, we have successfully done this in a server job. Our goal is to make this work in a parallel job.

We have not tried using 000 as field and record delimiter. I'll try that tomorrow. Would you expect "000" to indicate no delimiter?

Yes