Tracking count of records in a flat file

[Akumar1]

We have a requirement like this:

The first job extracts records from a data source and stores it in a flat file. The second job extracts these records from the file and performs transformations and then writes the transformed records to another flat file. Third job extracts the records from the last generated flat file and writes them to the target table. At the end of the sequencer run, there should be a flat file which holds the records extracted in the first job,records transformed in the second job and records inserted in the third job. What are all the various ways DataStage provides to implement this.

Can the LinkCount property be used to fetch and write the count of records to another sequential file after each job run, which will be called the audit file?

Any responses would greatly be appreciated.
Regards,
Akumar1

[ray.wurlod]

Yes it could, but you could simply create a second output from the Transformer stage that runs through an Aggregator stage (with no grouping columns) to produce the count.

[jdmiceli]

You could also use an after job ExecSH command to do a quick line count.
Something along the lines of :
wc -l #filename# >> countfilename

Just a thought.

[sambit]

Well... I would suggest to have a new col in each job which is popualted as @outrownum. You can later combine the @outrownum from each file and put it in a single file which will say how many rows were transformed in each of the job.