Page 1 of 1

Removing Header and Trailer record without using UNIX

Posted: Sun Jan 27, 2013 11:26 pm
by shantcha
Hello,
I want to remove header and trailer record from sequential file. As the datastage server is windows server, I am not considering Unix commands for this. I am able to import the matadata of the file, but when I am tryng to view the data through sequential file stage with the same metadata, it is giving import error at record 0. If I am able to read the data then I can remove header and trailer records through constraints.
There are total 13 columns in detail record, 4 columns in header record and only one column in trailer record. The file has pipe delimiter. The error it is giving is 'consumed more than 100000 bytes looking for record delimiter; aborting'.
Can any one help me

Posted: Sun Jan 27, 2013 11:40 pm
by chetan.c
Search for the exact error message and you will find the solution.

Although I suggest looking at file closely as it says ,it is unable to find the record delimiter to ensure the delimiter is present only after the
100000 bytes .

For trailer records , the sequential will reject the record automatically as it will not be matching the metadata.

Posted: Mon Jan 28, 2013 12:21 am
by ray.wurlod
There are effectively two strategies.

Use a reject link from the Sequential File stage to capture the header and trailer rows, and process them appropriately in a separate stream (even in the same job). You can join the header/trailer information back to the detail information later if desired.

Read the file as a single column (a suitably sized VarChar data type), use a Transformer stage to filter the header, detail and trailer rows to different output links, and process away.

I suspect you need to specify Record Delimiter String = DOS Style in your Sequential File stage in either case.