Page 1 of 1

Record Count in Transformer

Posted: Thu Jan 12, 2012 11:50 pm
by sg33
Hi ,
I have the job like ....
seq. file.....>transformer......>dataset

I want to count the number of records coming from the source .How can I do that in the transformer stage .

P.S :I don't want to use DSgetlinkinfo() and Aggregator stage or any script

Posted: Fri Jan 13, 2012 1:11 am
by chulett
Count them... for what purpose? To what end?

Posted: Fri Jan 13, 2012 1:53 am
by ray.wurlod
System variable @INROWNUM contains this count. Note that it is per-node.
You can not get the final count until the final row has been processed (on the current node).

Re: Record Count in Transformer

Posted: Sun Jan 15, 2012 5:21 am
by PhilHibbs
sg33 wrote:P.S :I don't want to use DSgetlinkinfo() and Aggregator stage or any script
Then you are pretty much out of luck. Aggregator is the way to do this. Does each row need to know the row count, for instance for computing averages, or a sequence string like "1 of 1000", "2 of 1000" etc., or do you just want one row output with the count? Why do you not want to use an Aggregator?

There's no way for the Transformer to know that the current row is the last one. You could write a routine to write the @INROWNUM to a file in overwrite mode and then pick up the value afterwards, then read the file. The Transformer would have to run in Sequential mode, not Parallel.

Re: Record Count in Transformer

Posted: Sun Jan 15, 2012 9:02 am
by chulett
PhilHibbs wrote:There's no way for the Transformer to know that the current row is the last one.
Actually, that was added in the 8.5 release, from what I recall.

As you noted, we'd still like to know the "why" of all this, then perhaps we can give a cogent answer.

Posted: Sun Jan 15, 2012 4:07 pm
by qt_ky
It's the LastRow() function in 8.5.

Posted: Tue Jan 17, 2012 3:27 am
by PhilHibbs
qt_ky wrote:It's the LastRow() function in 8.5.
Nice to know. Thanks.

Posted: Tue Jan 17, 2012 3:34 am
by v2kmadhav
Would doing a count and passing it as a param to this job be something that interests you ?

Posted: Tue Jan 17, 2012 3:40 am
by PhilHibbs
v2kmadhav wrote:Would doing a count and passing it as a param to this job be something that interests you ?
That would involve scripting, or at least shell, we do that with

Code: Select all

wc -l
but the OP is on windows.

Posted: Tue Jan 17, 2012 4:25 am
by v2kmadhav
Sorry my bad.. Didnt see the OS.

not sure about the reqmt but.. Since you using the sequential file... you could add the "rowNumberColumn" option and see if that helps you..

Posted: Tue Jan 17, 2012 7:36 am
by chulett
The included MKS Toolkit would allow the use of UNIX commands.

I'd still recommend waiting for the OP to come back and clarify their needs before people keep playing this endless guessing game.