Transformer
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 175
- Joined: Tue Oct 14, 2008 1:48 pm
Transformer
Hi,
Ca any body tell me how to count the number of records in a transformer.
For example I am sending 100 rows into the transformer and I want the output to be 1 row only having the count as 100.
Thanks
Ca any body tell me how to count the number of records in a transformer.
For example I am sending 100 rows into the transformer and I want the output to be 1 row only having the count as 100.
Thanks
Rajesh Kumar
The Transformer stage cannot be used for your intended purpose, because there is no way to detect that the last row is being read. As mentioned before, an Aggregator stage will do that.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Premium Member
- Posts: 151
- Joined: Fri Feb 13, 2009 4:19 pm
-
- Premium Member
- Posts: 175
- Joined: Tue Oct 14, 2008 1:48 pm
[quote="Mike"]Why transformer stage? Much easier when you use the sta
yes we can do that in aggregator stage, but if I use some functions already in the transformer and I want to implement this in the same transformer and dont want to add an extra stage.
Can we handle this in transformer???
yes we can do that in aggregator stage, but if I use some functions already in the transformer and I want to implement this in the same transformer and dont want to add an extra stage.
Can we handle this in transformer???
Rajesh Kumar
For the second time in this thread, it cannot be done in a transformer becuase you never know when the last row has arrived.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Premium Member
- Posts: 175
- Joined: Tue Oct 14, 2008 1:48 pm
-
- Premium Member
- Posts: 175
- Joined: Tue Oct 14, 2008 1:48 pm
Yes. We can count the number of rows in transformer but you will get 100 rows with counts as 1 to 100.kittu.raja wrote:Mike wrote:And don't worry about extra stages. That's what pipeline parallelism is all about...
And just for knowledge I am asking cant we count the number of rows in atranformer?? I thought we can do any transformations or process in the transformer.
Thanks
The only key point here is your requirement to get only 1 row with count as 100. That is where the problem is.
The only way to achieve this is through transformer constraint. But to derive this constraint we need to know the row count before the transformer either through aggregator or the count should be static which is not in your case.
Ping me if I am wrong...
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
I don't know why I am writing this
but I think its just to let you know how can you do that (is it for an interview?)
you need two transformers running in sequential mode
1. Using stage variable count the number of records and put it in a derived column say Count or use @OUTROWNUM
2.sort the data before second transformer using link sort on Count.
3. put a constraint in second transformer to release only one record(first) and pass only one column called count.
I don't think any one will use this technique except explaining it in an interview as this will create a slow running job ignoring the advantages of PX.
but I think its just to let you know how can you do that (is it for an interview?)
you need two transformers running in sequential mode
1. Using stage variable count the number of records and put it in a derived column say Count or use @OUTROWNUM
2.sort the data before second transformer using link sort on Count.
3. put a constraint in second transformer to release only one record(first) and pass only one column called count.
I don't think any one will use this technique except explaining it in an interview as this will create a slow running job ignoring the advantages of PX.
Priyadarshi Kunal
Genius may have its limitations, but stupidity is not thus handicapped.
Genius may have its limitations, but stupidity is not thus handicapped.