To get Record count in Transformer

[ravireddy25]

Hi,
How to get Record count in Transformer , i dont want to use aggregator, iam trying but not able to get the record count,is there any way to find the count ,i think we can do it through routine activity can any one suggest me how to do that ,bec till now i didnt use any routines .

Thanks in Advance

[ray.wurlod]

@INROWNUM

[PhilHibbs]

@INROWNUM

Well, that will give you the record count so far, and you could write out the previous @INROWNUM on a key change to get the count for a key, but you'll miss the last key group. Also, it only gives you the count in this partition. So, no, there isn't really a way. You could reduce the cost of the Aggregator Stage by just passing the key fields into it.

[ravireddy25]

@INROWNUM ...

@INROWNUM gives the incremental number for each record, but i want the count of the records coming from source

[Sainath.Srinivasan]

What is your source ?

As you have mentioned this as Server, is the source a sequential file or a table ?

If table, you can do a select count upfront.

If a sequential file, you can try something like

Code: Select all

type filename | find/c " "

[ravireddy25]

What is your source ?

As you have mentioned this as Server, is the source a sequential file or a table ?

If table, you can do a select count upfront.

If a sequential file, you can try someth ...

Hi,
Thanks for your reply ,Can you plz paste it , as i dont have premium account.

[ray.wurlod]

i dont have premium account.

That is easily remedied.

[chulett]

@INROWNUM

Well, that will give you the record count so far, and you could write out the previous @INROWNUM on a key change to get the count for a key, but you'll miss the last key group. Also, it only gives you the count in this partition. So, no, there isn't really a way. You could reduce the cost of the Aggregator Stage by just passing the key fields into it.

Server job, Phil. Plus there's been no mention of any 'key group' that I can see.

[chulett]

@INROWNUM ...

@INROWNUM gives the incremental number for each record, but i want the count of the records coming from source

You will need to use an Aggregator to get this number "in job" and really shouldn't be a big deal. Otherwise, you'll need to get the count either before or after job as noted and there are several ways to accomplish that.

[PhilHibbs]

There is no way to get a record count and make it available while processing each record other than by counting all the records first. If you don't do it in an Aggregator in DataStage, then you need to do it outside of DataStage with some kind of shell command like wc -l.

[chulett]

That's a valid point - we have no clue how the OP wants to actually use this total and that answer can change the mechanism needed for this.

[arunkumarmm]

@INROWNUM

Well, that will give you the record count so far, and you could write out the previous @INROWNUM on a key change to get the count for a key, but you'll miss the last key group. Also, it only gives you the count in this partition. So, no, there isn't really a way. You could reduce the cost of the Aggregator Stage by just passing the key fields into it.

Does it really works like this in Server? We normally use a stage variable as svCnt = svCnt +1 and assiging the initial value as 0

[chulett]

Phil posted that answer when the question was still over in the PX forum.

[mhester]

If you want to get a sequential count of rows processed regardless of the configuration file used then -

Code: Select all

@PARTITIONNUM + (@NUMPARTITIONS * (@INROWNUM-1))

will do the trick. If you want the total number of rows sourced then this can work, but an agg with a dummy column will output 1 row. You could source the table with a count(1) or some such thing and make it a reference to a lookup or something and then you will have the number.

Bunch of different ways to skin this cat with standard stages/ops

[chulett]

^
+---- In a PX job, of course.