Consumed more than 100000 bytes looking for record delimite

rsunny · Post by **rsunny** » Wed Jun 06, 2012 4:44 pm

Hi,

When i try to run the job i am getting the error as "Consumed more than 100000 bytes looking for record delimiter; aborting".

Source is Sequential File and the Final Delimiter : End , Field Delimiter:comma , Null Field Value:" and Quote: double.

I have created a user defined variable $APT_MAX_DELIMITED_READ_SIZE and defined at the job level to 300000 but still the job got Aborted . Can any one please provide me the solution for the issue

chulett · Post by **chulett** » Wed Jun 06, 2012 6:54 pm

What is your record delimiter? You did not mention that and that is what it is saying that it could not find.

rsunny · Post by **rsunny** » Wed Jun 06, 2012 8:31 pm

Hi craig,

I mentioned record delimeter as Unix new line and ran the job but still got aborted

SURA · Post by **SURA** » Wed Jun 06, 2012 8:36 pm

Use reject link in the Sequential file stage. It will help you to track where the issue is!

chulett · Post by **chulett** » Wed Jun 06, 2012 8:42 pm

If you used 'UNIX newline' and it couldn't find it, then that's not your record delimiter. What does a "wc -l" on your filename return?

ray.wurlod · Post by **ray.wurlod** » Wed Jun 06, 2012 8:43 pm

Apparently there is no UNIX newline characters in the first 100000 bytes (or 300000 bytes) of your file. It may, for example, be a fixed-width format file with no record delimiters at all. DataStage can handle that - but you have to set the Record Delimiter property to None.

chulett · Post by **chulett** » Wed Jun 06, 2012 9:02 pm

... if so then the answer to my question would be "1".

rsunny · Post by **rsunny** » Thu Jun 07, 2012 6:46 am

Hi ,

When i do wc -l filename i got the value as 3028799 .

Even though if i use a reject link for the Sequential stage , the job is getting Aborted.

Is there any possible solution to reject that record instead of Aborting the job?

chulett · Post by **chulett** » Thu Jun 07, 2012 7:34 am

You need to determine what the details of your file are so you can read it properly. A hex editor can help or perhaps "od -h" or some other flavor of a dump so you can see the actual hex/octal/decimal values.

zulfi123786 · Post by **zulfi123786** » Thu Jun 07, 2012 8:37 am

could you please share what is your actual record size based on column metadata ?