Remove Header and Trailor records from a fixed width file

[devidotcom]

Hi All,

I have gone through the posts here for removal of header and trailor records from a file.
I found that these posts has a requirement where the header and trailor were of different format hence we can automatically reject the records while reading the file.

I have a fixed width file where in the header and trailor needs to be removed or ignored while reading the data from the file.
The record length of these two records are the same as the file.
For example of the record length each record is 261 then the record length of header and trailor is the same with spaces appended at the end of them.

I am aware that we can remove the header and trailor via a script but we are looking at implementing it in DS.

Any help on this? Thanks in advance.

[ray.wurlod]

Is there any information in any of the fields that indicates the record type as header, detail, trailer? If so, simply use that value in the constraint expression of a Transformer or Filter stage.

[Govindarajan]

If you do not have a clue about the header or footer follow the process:-

a) write a shell script to get the number of lines (use wc -l <filename>) and store this value in a text file
b) call this shell script from before job routine
c) in your constraint do not process the following conditions

either first record or your inrownum is equivalent to the value from the text file

[devidotcom]

Thanks for reply.

I don't want to use a script.
Yes, Ray the header looks like
1107081700153721HEADER DPCUSCOM

and trailor looks like
9000000000051853TRAILER

But both these have length same as the record lenght with spaces appended at the end.
We can identify the record based on the words HEADER and TRAILOR but how I mention the field.
For example the word header is at 17 to 21 position. I may have record column coming from 10-20 .
How do I add the constraint

[ray.wurlod]

You could read the entire record as a single string, then use the Index() function to test for the existence of "HEADER" or "TRAILER".

Or you need to make sure that your column width definitions properly define the seven character field in which the record type is to be found.