Removing Header and Trailer record without using UNIX

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
shantcha
Participant
Posts: 1
Joined: Tue Jul 14, 2009 11:55 pm

Removing Header and Trailer record without using UNIX

Post by shantcha »

Hello,
I want to remove header and trailer record from sequential file. As the datastage server is windows server, I am not considering Unix commands for this. I am able to import the matadata of the file, but when I am tryng to view the data through sequential file stage with the same metadata, it is giving import error at record 0. If I am able to read the data then I can remove header and trailer records through constraints.
There are total 13 columns in detail record, 4 columns in header record and only one column in trailer record. The file has pipe delimiter. The error it is giving is 'consumed more than 100000 bytes looking for record delimiter; aborting'.
Can any one help me
chetan.c
Participant
Posts: 112
Joined: Tue Jan 17, 2012 2:09 am
Location: Bangalore

Post by chetan.c »

Search for the exact error message and you will find the solution.

Although I suggest looking at file closely as it says ,it is unable to find the record delimiter to ensure the delimiter is present only after the
100000 bytes .

For trailer records , the sequential will reject the record automatically as it will not be matching the metadata.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

There are effectively two strategies.

Use a reject link from the Sequential File stage to capture the header and trailer rows, and process them appropriately in a separate stream (even in the same job). You can join the header/trailer information back to the detail information later if desired.

Read the file as a single column (a suitably sized VarChar data type), use a Transformer stage to filter the header, detail and trailer rows to different output links, and process away.

I suspect you need to specify Record Delimiter String = DOS Style in your Sequential File stage in either case.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply