Data file having header and taielr. Want to read the file wi

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mallikachaithu
Participant
Posts: 4
Joined: Thu Jul 22, 2010 4:38 am

Data file having header and taielr. Want to read the file wi

Post by mallikachaithu »

Hi,

I have a file which has header in 1st row and tailer in last row.
ex:
" This is a test desciption
1,120,hyd
2,250,che
3,350,Bang
total is 720 "
in the above file 1st line is header, last line is tailer. in between is the data.
Now i want to read only data(ie header, tailer and data seperately) using sequential file. Do some calculations and finally dump in the target same as the source file. ie 1st line as header, last line as tailer and in between data.

Could any pls help me how can i do this.

Thanks a ton in advance.
Mallika
Mallika
ds@tcs
Participant
Posts: 24
Joined: Thu Mar 17, 2011 6:26 am
Location: chennai
Contact:

Re: Data file having header and taielr. Want to read the fil

Post by ds@tcs »

All the detail records will be having same index, header and trailer will be having different index
like
11|25/01/2011
22|A|123|434
22|B|345|567
-
-
22|H|545|858
99|9(record count)
so while extracting the data from source u can extract only the records which are starting with 22
Regards
Sankar
tbharathkumar
Participant
Posts: 26
Joined: Mon Aug 27, 2007 6:27 am
Location: Des Moines

Re: Data file having header and taielr. Want to read the fil

Post by tbharathkumar »

have a unix script to just copy source file in before job sob routine, using SED or unix shell script help keep 1 & last line , remove rest. do all your calucations, in after job subroutine again create simple Copy script or ftp script to transfer files....:-)
Regards,
Bharath Tipirisetty
MarkB
Premium Member
Premium Member
Posts: 95
Joined: Fri Oct 27, 2006 9:13 am

Post by MarkB »

In a Before Job routine, do an ExecSh and head -1 the sourcefile to one file and tail -1 the sourcefile to another. Let your job run as-is; the header and trailer rows will drop off as rejects because of not meeting the file specifications. In an After Job routine, do an ExecSH and cat the header file, the data file you created in your job, and the trailer file and output to the file name/location of your choice.
Sampath
Participant
Posts: 21
Joined: Tue Apr 05, 2005 2:03 am

Re: Data file having header and taielr. Want to read the fil

Post by Sampath »

Use UNIX command before us the file
sed "1d;\$d" <FileName>

It will remove the header and trailer records from file
Regards,
Sampath
Sampath
Participant
Posts: 21
Joined: Tue Apr 05, 2005 2:03 am

Re: Data file having header and taielr. Want to read the fil

Post by Sampath »

Use UNIX command before us the file
sed "1d;\$d" <FileName>

It will remove the header and trailer records from file
Regards,
Sampath
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Re: Data file having header and taielr. Want to read the fil

Post by ray.wurlod »

Sampath wrote:Use UNIX command before us the file
sed "1d;\$d" <FileName>

It will remove the header and trailer records from file
What if they need the contents of the header and trailer?

Very poor advice, in my opinion.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Post by zulfi123786 »

One approach of using just a job without scripts and sequencers is to read the file as a single column per record use a stage variable to count the number of delimiters which you would use to identify header, trailer and data records. Use in row number to identify header and flush into one link all data columns into one link using the stage variable as mentioned with index function in it and here you use the column import stage to divide the data into columns and do your processing , then in the third link (otherwise link ) will hold your trailer and finally use a ordered funnel to get back the header , processed data and trailer.

The excersice is Labourios - I accept
Post Reply