Getting record count from Transformer

[aaikat]

I have 2 source tables followed by Join stage(outer join),followed by Transformer stage which branches data into 4 output sequential files.
Now I want to calculate the total records coming into Transformer stage and write it to a sequential file.
I used a stage variable 'TotalCounts' in Transformer, set its initial value to 0 and in derivation 'TotalCounts + 1'
But when I am looking at the output the TotalCounts values written to the file gets recycled after some interval. Please suggest how it can be done.

[ray.wurlod]

System variable @INROWNUM maintains the count of rows on the input link, while each output link maintains its own @OUTROWNUM (on each partition in each case). You could collect these into an extra two columns on each output link, and use downstream Copy stages to split them out, and Aggregator stages (perhaps in Sequential mode) to report the Max of each.

[aaikat]

I am not clear about how to collect these.Will you plz elaborate.
I dbout due to partitioning I am not getting the correct count.
say there are 1000 input record counts
the variable doesn't show 1,2,3....,1000
Rather it is showing 1,2,3,...100,1,2,3,....
How to get rid of that.

[richdhan]

Hi aaikat,

After join stage have a copy stage to branch the records to a transformer and an aggregator(in sequential mode) to get the record count.

HTH
--Rich

[kumar_s]

Are you getting the numbers for each partition, then you should be able to make up with the combinations of followings @PARTITIONNUM,@NUMPARTITIONS,@INROWNUM

[kool78]

Hi

Set the Transformer mode to Sequential and assign the system variable @INROWNUM to the column which u want to hold the count, then aggregate upon the column and get the max of it, that would be your number of records from source.

[prabu]

I have 2 source tables followed by Join stage(outer join),followed by Transformer stage which branches data into 4 output sequential files.
Now I want to calculate the total records coming into Transformer stage and write it to a sequential file.
I used a stage variable 'TotalCounts' in Transformer, set its initial value to 0 and in derivation 'TotalCounts + 1'
But when I am looking at the output the TotalCounts values written to the file gets recycled after some interval. Please suggest how it can be done.

have you tried DSGetLinkInfo (JobHandle, StageName, LinkName, InfoType)??