check for a particular row existing in the incoming file

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
hiltsmi
Participant
Posts: 20
Joined: Thu Aug 04, 2005 9:03 am

check for a particular row existing in the incoming file

Post by hiltsmi »

The first 3 characters of each incoming record indicates the record type. Record type = '999' indicates that the record is the trailer record. At the end of each incoming file there should be a trailer record, which contains a count of the number of records in the file.

I have a transformer stage in which I want to compare the record count in the trailer record to the link count and abort if they don't match. When the file contains a trailer record I can find it and I can successfully determine if the record is correct.

The problem is when the trailer record is missing. If I don't see the trailer record in the file I also want to abort. How can I tell the transformer to abort if the end of data is reached without finding the trailer record?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

A much better solution is to check for this before processing the file, not least because you will end up doing a lot less processing.
Use the UNIX command tail -1 filename to report the last line of the file, and use something like grep ^999 to determine whether the line begins with 999. Pipe the output of tail to the input of grep, and test to see whether there is a line in the output of grep.
Since you're on Windows you need MKS Toolkit (which ships as part of DataStage 7.5x2 at least) or some other "UNIX on Windows" kit.
The entire command could be executed from an Execute Command activity in the controlling job sequence.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Hi,
in server edition, option to get the tail record is limited, (unlike parallel TAIL stage) so we need to externelize it, so i guess ray's suggestion wud be the best.

regards
kumar
Post Reply