To get Record count in Transformer
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 59
- Joined: Wed Dec 31, 2008 5:49 am
To get Record count in Transformer
Hi,
How to get Record count in Transformer , i dont want to use aggregator, iam trying but not able to get the record count,is there any way to find the count ,i think we can do it through routine activity can any one suggest me how to do that ,bec till now i didnt use any routines .
Thanks in Advance
How to get Record count in Transformer , i dont want to use aggregator, iam trying but not able to get the record count,is there any way to find the count ,i think we can do it through routine activity can any one suggest me how to do that ,bec till now i didnt use any routines .
Thanks in Advance
Ravi
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
Well, that will give you the record count so far, and you could write out the previous @INROWNUM on a key change to get the count for a key, but you'll miss the last key group. Also, it only gives you the count in this partition. So, no, there isn't really a way. You could reduce the cost of the Aggregator Stage by just passing the key fields into it.ray.wurlod wrote:@INROWNUM
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
-
- Participant
- Posts: 59
- Joined: Wed Dec 31, 2008 5:49 am
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
What is your source ?
As you have mentioned this as Server, is the source a sequential file or a table ?
If table, you can do a select count upfront.
If a sequential file, you can try something like
As you have mentioned this as Server, is the source a sequential file or a table ?
If table, you can do a select count upfront.
If a sequential file, you can try something like
Code: Select all
type filename | find/c " "
-
- Participant
- Posts: 59
- Joined: Wed Dec 31, 2008 5:49 am
Sainath.Srinivasan wrote:What is your source ?
As you have mentioned this as Server, is the source a sequential file or a table ?
If table, you can do a select count upfront.
If a sequential file, you can try someth ...
Hi,
Thanks for your reply ,Can you plz paste it , as i dont have premium account.
Ravi
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Server job, Phil. Plus there's been no mention of any 'key group' that I can see.PhilHibbs wrote:Well, that will give you the record count so far, and you could write out the previous @INROWNUM on a key change to get the count for a key, but you'll miss the last key group. Also, it only gives you the count in this partition. So, no, there isn't really a way. You could reduce the cost of the Aggregator Stage by just passing the key fields into it.ray.wurlod wrote:@INROWNUM
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
You will need to use an Aggregator to get this number "in job" and really shouldn't be a big deal. Otherwise, you'll need to get the count either before or after job as noted and there are several ways to accomplish that.ravireddy25 wrote:@INROWNUM gives the incremental number for each record, but i want the count of the records coming from sourceray.wurlod wrote:@INROWNUM ...
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
There is no way to get a record count and make it available while processing each record other than by counting all the records first. If you don't do it in an Aggregator in DataStage, then you need to do it outside of DataStage with some kind of shell command like wc -l.
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
-
- Participant
- Posts: 246
- Joined: Mon Jun 30, 2008 3:22 am
- Location: New York
- Contact:
Does it really works like this in Server? We normally use a stage variable as svCnt = svCnt +1 and assiging the initial value as 0PhilHibbs wrote:Well, that will give you the record count so far, and you could write out the previous @INROWNUM on a key change to get the count for a key, but you'll miss the last key group. Also, it only gives you the count in this partition. So, no, there isn't really a way. You could reduce the cost of the Aggregator Stage by just passing the key fields into it.ray.wurlod wrote:@INROWNUM
Arun
If you want to get a sequential count of rows processed regardless of the configuration file used then -
will do the trick. If you want the total number of rows sourced then this can work, but an agg with a dummy column will output 1 row. You could source the table with a count(1) or some such thing and make it a reference to a lookup or something and then you will have the number.
Bunch of different ways to skin this cat with standard stages/ops
Code: Select all
@PARTITIONNUM + (@NUMPARTITIONS * (@INROWNUM-1))
Bunch of different ways to skin this cat with standard stages/ops
Mike Hester
mhester@petra-ps.com
mhester@petra-ps.com