Need to remove header and footer in Source Sequential File

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
cosec
Premium Member
Premium Member
Posts: 230
Joined: Tue May 08, 2007 8:10 pm

Need to remove header and footer in Source Sequential File

Post by cosec »

The source is a Fixed width sequential file. The footer contains the number of records in the file excluding the header and the footer.

I am required to :

(1.)To verify if the number of actual records in the file, match the number of records stated in the footer.

(2) To Remove the header and footer when loading.....

Would appreciate any suggestions to accomplish these tasks....

Thanks
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

You can tell the stage to ignore the first line (state that it contains the column names). If the footer has a different format you can split the logic in a transform stage and use the @INROWNUM stage variable to determine if the record count is correct.
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

To remove header and footer from your file use this command in your filter option of sequential file satge:

Code: Select all

sed -e '$d' -e '1d'
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Preprocess with a before-job subroutine. This can write a new file, without header or footer, as well as reconciling the row count and preventing job execution where the counts fail to reconcile.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
cosec
Premium Member
Premium Member
Posts: 230
Joined: Tue May 08, 2007 8:10 pm

Post by cosec »

I tried Using the following command.....sed -e '$d' -e '1d'...The job gets aborted and this is the message :

Load_Test..TBL_TEST.Input: ds_seqopen() - Win32 error in CreateProcess() - The system cannot find the file specified.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

And if you remove the sed command from the filter does the job work?
cosec
Premium Member
Premium Member
Posts: 230
Joined: Tue May 08, 2007 8:10 pm

Re: Need to remove header and footer in Source Sequential F

Post by cosec »

I got it working...Thanks A lot....I was trying it on a windows platform that's why it wasnt working.....




cosec wrote:The source is a Fixed width sequential file. The footer contains the number of records in the file excluding the header and the footer.

I am required to :

(1.)To verify if the number of actual records in the file, match the number of records stated in the footer.

(2) To Remove the header and footer when loading.....

Would appreciate any suggestions to accomplish these tasks....

Thanks
Post Reply