skip header records in a sequential file

Aggie99 · Post by **Aggie99** » Tue Apr 07, 2009 12:34 am

hi guys,

I need your help on how to skip reading the first 10 rows of a sequential file.

I have a csv file, the first 10 records are metadata, the rest of the file is data records.

Metadata have different field format from non-metadata.

I try to read in the data, but not the metadata using sequential file stage. But I don't see an option that let me SKIP the first XX records so I can read from row 11 onward.

What can I do? thx for your help.
-Aggie

Kryt0n · Post by **Kryt0n** » Tue Apr 07, 2009 12:53 am

Can you run Unix commands on a Windows DS server? (we can on our clients so would expect similar functionality on the server)

An alternative is to define the input as one column (type varchar) per row and drop the first 10 rows through a transformer or filter stage

chulett · Post by **chulett** » Tue Apr 07, 2009 7:40 am

Define the columns in the stage to match the data columns in the file, the records from 11 on. The stage will automatically discard the first 10 for not matching that.

Aggie99 · Post by **Aggie99** » Tue Apr 07, 2009 8:38 am

that's what I am doing now, define the columns for records from row 11 onward. This resulted in a set of warnings in the log.
I guess I can downgrade the warnings. But I was hoping there're other options.

thx guys.

aggie

ray.wurlod · Post by **ray.wurlod** » Tue Apr 07, 2009 9:32 am

Put a reject link on the Sequential File stage. It will capture any row that does not match the metadata on the main output link.