Page 1 of 1
Header & Trailer Record Warning
Posted: Mon Aug 28, 2006 11:25 am
by nvalia
Hi,
I get the following warning message when I try to read from the Source Sequential file "SEQ_Histpion,0: Delimiter for field "Field004" not found; input: {0d}, at offset: 107"
The reason is that the Header (and also Trailer) record do not have same metadata as all other data rows i.e. data rows have 28 cols where as these 2 (header & footer) have only 3 cols
How to do away with this warning.
Thanks,
NV
Posted: Mon Aug 28, 2006 11:45 am
by Krazykoolrohit
To ignore the header, check "First line contains column names" check box in 'output->format' tab.
I dont know how you can remove a trailor with datastage.
Sinc you are in unix, you can try removing them by using a filter command or writing a Script which can be called through "before job subroutine"
Posted: Mon Aug 28, 2006 4:08 pm
by mansoor_nb
Use the Filter option in the Format tab.If your Header, Detail and Trailer records are identified by the Record type then you can use the Grep command in the filter in the Format tab to read the detail records.
Posted: Tue Aug 29, 2006 7:14 am
by nvalia
Hi,
We do not get any record type identifier.We simply get a header rec as the first record and the Trailer rec as the second last record with an empty line as the last one.
Posted: Tue Aug 29, 2006 8:08 am
by thumsup9
What Mansoor said is a good option. If we have a record identifier tagged to header and trailer records, you can grep and get just the detail records. What about skipping the rows if you know the number of header and trailer rows.
Posted: Tue Aug 29, 2006 8:25 am
by Krazykoolrohit
why dont you just cut off the first and last rows coding a shell script as i suggested earlier?
Posted: Tue Aug 29, 2006 6:07 pm
by kumar_s
Code: Select all
head -$(expr $(wc -l filename | awk '{ print $1 }') \- 1) filename | tail +2
Use this in Beforejob subroutine or in FilterCommand in you sequential stage to strip out both header and trailer.
Posted: Tue Aug 29, 2006 8:44 pm
by chulett
Make sure your O/S supports positive offsets with the 'head' function if you are thinking about doing this. HP-UX 11 does not, for example.
Posted: Tue Aug 29, 2006 10:55 pm
by ray.wurlod
Also not all UNIX operating systems/emulators support positive arguments for the tail command.
Posted: Fri Sep 01, 2006 2:31 pm
by nvalia
Thanks to All..
I will test it out
Posted: Fri Sep 01, 2006 9:19 pm
by ray.wurlod
Use a rejects link from the Sequential File stage to capture the header and trailer row (as raw strings) into a separate data stream. You can process them separately or just ignore them.
Posted: Tue Sep 05, 2006 1:48 pm
by nvalia
Hi,
I had already tried it, but it still throws the warnings. It seems writting a Unix script for this might be the only option.
Thanks,
Nirav