How do you handle break processing in DataStage
Posted: Tue Feb 16, 2010 11:27 am
I've been investigating how to do break processing within DataStage but have had no luck... here is what I need to do...
From a single input source I need to group rows together by 3 key values (not a problem) then I need to assign an incrementing number starting from 1 and continuing to n based upon the number of rows the 3 key values present... so if keys 1-3 are identical for 5 records, I need to assign the first one with a '1', the 2nd with a '2', and so on... and for each gouping of the 3 keys, I need to start with '1' again...
I've looked at sequence generator, aggregator, stage variables, etc... and have not figured out how to maintain the previous values, or set up a variable 'counter' that I can reset to '1' each time I get new key values...
Am I missing something in DS???... does a stage exist that can do this???... or has anyone had a similar situation and had to do a workaround...
Any help would be appreciated...
From a single input source I need to group rows together by 3 key values (not a problem) then I need to assign an incrementing number starting from 1 and continuing to n based upon the number of rows the 3 key values present... so if keys 1-3 are identical for 5 records, I need to assign the first one with a '1', the 2nd with a '2', and so on... and for each gouping of the 3 keys, I need to start with '1' again...
I've looked at sequence generator, aggregator, stage variables, etc... and have not figured out how to maintain the previous values, or set up a variable 'counter' that I can reset to '1' each time I get new key values...
Am I missing something in DS???... does a stage exist that can do this???... or has anyone had a similar situation and had to do a workaround...
Any help would be appreciated...