2 header rows and one tail row should be removed

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kkumardatastage
Participant
Posts: 84
Joined: Sat Jul 19, 2008 8:50 am

2 header rows and one tail row should be removed

Post by kkumardatastage »

hi
i got a file with one header row and one tail row in between this there is a data. and we need to remove the 2 headers and the 1 tail row. the data which is in between 2 headers and 1 tailer should be used as a source sequential file and the target is also a sequential file. and i am using datastage parallel job. but i need to get the 071221 as a new column as (field6). and i am getting this type of file in daily base.
can any one help me that this type of file can be taken in datastage parallel. pls let me know how can i do this job .

for ex:
i/p file

! file created on 071221
! date 071221
field1 field2 field3 field4 field5
data data data data data
data data data data data
data data data data data
data data data data data
data data data data data
data data data data data
all rows 7.

o/pfile
field1 field2 field3 field4 field5 filed6
data data data data data 071221
data data data data data 071221
data data data data data 071221
data data data data data 071221
data data data data data 071221
data data data data data 071221



thank you
filename.txt
Participant
Posts: 27
Joined: Thu Mar 20, 2008 11:55 am

Post by filename.txt »

Easy way would be removing these two Header rows and tails row in unix using a shell script.
-------------------------------------------------------------------------
#!/bin/ksh
D=`head -1 FILENAME|tr -dc '[a-z]'`
a=`cat FILENAME|wc -l`
b=`expr $a - 3`
c=`expr $a - 1`
for [ i -eq $b ]
do
echo $D >>DATEFILE.txt
done
awk 'NR>=2&&NR<=$c' "FILENAME" > NEWFILENAME
paste -d, NEWFILENAME DATEFILE > FILENAME
rm DATEFILE
rm NEWFILENAME
-------------------------------------------------------------------------
I couldn't get 'for' statement working... But consider this a pseudo code.

This is just a sample script....make sure you point to right directory & file.
Thanks.

"Creativity is the ability to use your available resources to their fullest."
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Read the sequential file using a schema that is correct for the detail lines. Add a Reject link - this will capture the headers and trailer because they do not have the same structure as the detail lines. The Reject link can then lead to other stages that can re-parse the captured string (maybe using a Column Import or Transformer stage) and do any required processing of these.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply