Page 1 of 1
Posted: Tue Dec 03, 2013 4:35 pm
by asorrell
How many rows? For small to medium sized row counts, the easiest way is to run the job in sequential mode and sort the data prior to entering the transformer. Just use @ROWNUM to generate your numbers.
Posted: Wed Dec 04, 2013 12:25 am
by deeplind07
Hash Partition (on F1 )and sort(and f1 and f3) in the input link of transformer.
Use following stage variables in the transformer.
Default value of all stage varibles should be 0
SVF1_1=F1
IF SVF11=SVF12 THEN SVF2= SVF2+1 ELSE 1
SVF12=SVF11
Posted: Tue Dec 10, 2013 7:31 am
by sec105105
deeplind07 wrote:Hash Partition (on F1 )and sort(and f1 and f3) in the input link of transformer. ...
SVF1_1=F1
IF SVF11=SVF12 THEN SVF2= SVF2+1 ELSE 1
SVF12=SVF11
Something similar to the above worked. Thanks!
Posted: Tue Dec 10, 2013 8:36 am
by chulett
Hopefully something similar but with better stage variable names.
FYI - this is old school stage variable usage for resetting a counter while doing group change detection. And the first variable isn't really needed but can be helpful if your 'group' consists of multiple fields. And as with all things Stage Variable, order matters as they evaluate top down.
Code: Select all
svCounter: If F1 = svOldF1 then svCounter+1 else 1
svOldF1 = F1
You can simplify the group check by letting a Sort stage do that for part for you. If you've done that you'll have a new field called
KeyChange that is set to True on the first entry of a group. So the code becomes:
Code: Select all
svCounter: If KeyChange then 1 else svCounter+1