Sequential file without a first line

[goffinw]

I have a sequential file where the first line is to be ignored.
However, it is not a title line, and the number of fields in the first line is smaller then in the following data lines. This last fact makes the use of the use of the Sequential File stage, and the option "First line is column names" not usable: an error is generated on the first line, as fields are missing.
What would be the best way to read a sequential file of which the first line is to be ignored.

Thanks,
Wim

[vigneshra]

You can use simple unix commands to cut the first line out of the file. That is the easiest way for doing this !!

[Orieos]

I would use a very simple unix command:

tail +2 inFile > outFile

then pass the outFile to Datastage...

Hope it helps

Ricardo

[chucksmith]

You can use these unix command as filters directly in the sequential stage. On the stage tab, check Stage uses filter commands, and on the Outputs / General tab enter the unix command in the Filter command: box.

[chulett]

You could still play the 'First line is column names' game. In addition to that, check off the 'Ignore row truncation warnings'... either that or slide the 'Columns' definition tab to the right and set the entries that control missing columns appropriately.

Or as noted in previous posts, use UNIX commands to remove the first record. I'd suggest doing this in the 'Filter' box on the Sequential File stage rather than messing with 'before' commands.

Edit: Chuck beat me to that one.

[goffinw]

I understand your answers, but they require that space is allocated for the intermediate file and that these files be handleded correctly.
A Named Pipe would also be a solution for the intermediate file, but this also needs to be set up and managed in a reliable way. That's not always easy to do.
All this 'overhead' this would be unnecessary if you remain within the Datastage Designer environment. I would prefer DataStage to take care of all the data piping.

Would you agree with that consideration?
Thanks,
Wim

[chulett]

No. The use of the Filter requires no intermediate files. Neither does the other options I outlined.

[goffinw]

Chuck,

I believe your answer is what I need to get the unix command into the pipe as I require.
I will try this out.

Thanks to all,
Wim

[chulett]

Ok. Read the online help for the Filter option. There's no need to use a named pipe for this, unless you prefer to take that route.

Simply putting the appropriate command in the Filter box (without the redirection as it leverages standard out) gets you what you need.

[Orieos]

I think that what chullet is trying to say is:

1.Activate the Filter command box.

2.Write in the box something like --> tail +2 <your input file name>

3. There is no 3....it's done :D

Hope it helps

Ricardo

[goffinw]

Thanks for all the help.
Your answers come faster then I can react to.

It suffied indeed to write the unix filter command 'tail +2' into the filter of the sequential file stage where I read the file.

Thanks,
Wim

[chulett]

3. There is no 3....it's done :D

Excellent... there is no 3.

Thanks for the clarification, and glad it got worked out.