Sequential File...footer record have different format

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
sudhirmah
Participant
Posts: 17
Joined: Sat Oct 23, 2004 5:07 am

Sequential File...footer record have different format

Post by sudhirmah »

Hi,

I have a sequential file having footer record differrent format.
in sequential stage getting warning on the last record.
any workaround instead of multiple record type handling trick.

TIA
SAM
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The answer depends upon the type of warning you are getting with the footer row - please tell us what the warning is. If it is a sequential-file read error there might not be a way to read that last errant row without generating a message, so you might need to execute a before-job command that strips out the last line, or does a "wc -l" to get the number of lines and use that value in your job.
sudhirmah
Participant
Posts: 17
Joined: Sat Oct 23, 2004 5:07 am

Post by sudhirmah »

Hi,

am getting the error in the sequential stage...as below...

SRC_SEQ,0: Delimiter for field "ASSET" not found; input: {0d}, at offset: 41
.
.
SRC_SEQ,0: Import complete. 1 records imported successfully, 1 rejected.

TIA
SAM
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

As you are using PX, stripping the last line will be the possibility. But using 'wc -l' may not be as the last record in the file may not be the last record processed.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Just saw the same thing, it is a PX job (but posted in the Server forum) so the solution is quite different. As Sainath stated, with a Px sequential file read you can specify "read first rows" to limit the number of rows read using the output from a "wc -l" UNIX command and using 1 node. The best solution if the input file isn't huge is to make a copy without the offending last line.
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Try using 'external source'.
bcarlson
Premium Member
Premium Member
Posts: 772
Joined: Fri Oct 01, 2004 3:06 pm
Location: Minnesota

Post by bcarlson »

Not sure if this is the 'best' way to do it, but it worked well for us. We had a file that had both header and footer records. On import, we specified the main record layout and then made sure there was a reject link. The reject from an import leaves the data as a raw field so on the reject link we used the field import stage. The field import used the layout for the header/footer (they happened to be the same layout). We also had a reject link coming out of the field import (for real rejected records) and those got dumped into a dataset.

If you just don't need the header/footer, you could just leave them as rejected records (and skip the field import).

bcarlson
Post Reply