Page 1 of 1

2 header rows and one tail row should be removed

Posted: Fri Aug 29, 2008 11:57 am
by kkumardatastage
hi
i got a file with one header row and one tail row in between this there is a data. and we need to remove the 2 headers and the 1 tail row. the data which is in between 2 headers and 1 tailer should be used as a source sequential file and the target is also a sequential file. and i am using datastage parallel job. but i need to get the 071221 as a new column as (field6). and i am getting this type of file in daily base.
can any one help me that this type of file can be taken in datastage parallel. pls let me know how can i do this job .

for ex:
i/p file

! file created on 071221
! date 071221
field1 field2 field3 field4 field5
data data data data data
data data data data data
data data data data data
data data data data data
data data data data data
data data data data data
all rows 7.

o/pfile
field1 field2 field3 field4 field5 filed6
data data data data data 071221
data data data data data 071221
data data data data data 071221
data data data data data 071221
data data data data data 071221
data data data data data 071221



thank you

Posted: Fri Aug 29, 2008 12:36 pm
by filename.txt
Easy way would be removing these two Header rows and tails row in unix using a shell script.
-------------------------------------------------------------------------
#!/bin/ksh
D=`head -1 FILENAME|tr -dc '[a-z]'`
a=`cat FILENAME|wc -l`
b=`expr $a - 3`
c=`expr $a - 1`
for [ i -eq $b ]
do
echo $D >>DATEFILE.txt
done
awk 'NR>=2&&NR<=$c' "FILENAME" > NEWFILENAME
paste -d, NEWFILENAME DATEFILE > FILENAME
rm DATEFILE
rm NEWFILENAME
-------------------------------------------------------------------------
I couldn't get 'for' statement working... But consider this a pseudo code.

This is just a sample script....make sure you point to right directory & file.

Posted: Fri Aug 29, 2008 5:25 pm
by ray.wurlod
Read the sequential file using a schema that is correct for the detail lines. Add a Reject link - this will capture the headers and trailer because they do not have the same structure as the detail lines. The Reject link can then lead to other stages that can re-parse the captured string (maybe using a Column Import or Transformer stage) and do any required processing of these.