removal of header and trailor from seq file

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ankitaporwal
Participant
Posts: 4
Joined: Mon Jul 23, 2007 3:47 am

removal of header and trailor from seq file

Post by ankitaporwal »

Hi All,

Please let me know how through Datastage job (parallel) we can remove the header and trailor in sequential file. If the header and trailor has same no of field as the records and also the delimiter is same as the record.
Is it done through HEAD /TAIL stage....
Please resolve my query.

Thanks,
Ankita
dhanashreepanse
Participant
Posts: 25
Joined: Fri Jan 11, 2008 12:49 am
Location: Pune, India

Post by dhanashreepanse »

Head/Tail stage can't be used here since you won't know the exact number of records in the file.

In the sequential file stage, go to Output --> Properties --> Options.
Set the filter property as: sed '$d' | sed '1,1d'
First one removes the trailer and the next one removes the header.

-Dhanashree
vrishabhsagar
Participant
Posts: 33
Joined: Mon Nov 12, 2007 1:02 am
Location: Bangalore

Re: removal of header and trailor from seq file

Post by vrishabhsagar »

ankitaporwal wrote:Hi All,

Please let me know how through Datastage job (parallel) we can remove the header and trailor in sequential file. If the header and trailor has same no of field as the records and also the delimiter is same as the record.
Is it done through HEAD /TAIL stage....
Please resolve my query.

Thanks,
Ankita
If the header has a identifiable format, [like it starts with 'HDR' or a particular col like 'Record type' is 1], then you may employ either filter stage or transformer to apply a constraint to knock down header and trailers. I have done this once with fixed length file, but I am a nooB with DS Px myself. Hope this helps.
Rishabh Sagar V
Bangalore
Post Reply