Header & Trailer Record Warning

nvalia · Post by **nvalia** » Mon Aug 28, 2006 11:25 am

Hi,

I get the following warning message when I try to read from the Source Sequential file "SEQ_Histpion,0: Delimiter for field "Field004" not found; input: {0d}, at offset: 107"
The reason is that the Header (and also Trailer) record do not have same metadata as all other data rows i.e. data rows have 28 cols where as these 2 (header & footer) have only 3 cols

How to do away with this warning.

Thanks,
NV

Krazykoolrohit · Post by **Krazykoolrohit** » Mon Aug 28, 2006 11:45 am

To ignore the header, check "First line contains column names" check box in 'output->format' tab.

I dont know how you can remove a trailor with datastage.

Sinc you are in unix, you can try removing them by using a filter command or writing a Script which can be called through "before job subroutine"

mansoor_nb · Post by **mansoor_nb** » Mon Aug 28, 2006 4:08 pm

Use the Filter option in the Format tab.If your Header, Detail and Trailer records are identified by the Record type then you can use the Grep command in the filter in the Format tab to read the detail records.

nvalia · Post by **nvalia** » Tue Aug 29, 2006 7:14 am

Hi,

We do not get any record type identifier.We simply get a header rec as the first record and the Trailer rec as the second last record with an empty line as the last one.

thumsup9 · Post by **thumsup9** » Tue Aug 29, 2006 8:08 am

What Mansoor said is a good option. If we have a record identifier tagged to header and trailer records, you can grep and get just the detail records. What about skipping the rows if you know the number of header and trailer rows.

Krazykoolrohit · Post by **Krazykoolrohit** » Tue Aug 29, 2006 8:25 am

why dont you just cut off the first and last rows coding a shell script as i suggested earlier?

kumar_s · Post by **kumar_s** » Tue Aug 29, 2006 6:07 pm

Code: Select all

head -$(expr $(wc -l filename | awk '{ print $1 }') \- 1) filename | tail +2

Use this in Beforejob subroutine or in FilterCommand in you sequential stage to strip out both header and trailer.

chulett · Post by **chulett** » Tue Aug 29, 2006 8:44 pm

Make sure your O/S supports positive offsets with the 'head' function if you are thinking about doing this. HP-UX 11 does not, for example.

ray.wurlod · Post by **ray.wurlod** » Tue Aug 29, 2006 10:55 pm

Also not all UNIX operating systems/emulators support positive arguments for the tail command.

nvalia · Post by **nvalia** » Fri Sep 01, 2006 2:31 pm

Thanks to All..
I will test it out

ray.wurlod · Post by **ray.wurlod** » Fri Sep 01, 2006 9:19 pm

Use a rejects link from the Sequential File stage to capture the header and trailer row (as raw strings) into a separate data stream. You can process them separately or just ignore them.

nvalia · Post by **nvalia** » Tue Sep 05, 2006 1:48 pm

Hi,

I had already tried it, but it still throws the warnings. It seems writting a Unix script for this might be the only option.

Thanks,
Nirav