Picking first 11 records in a Transformer

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Suman
Participant
Posts: 53
Joined: Thu Oct 07, 2004 8:28 am

Picking first 11 records in a Transformer

Post by Suman »

I have records coming like this into a transformer:

Id1 Id2 Field
123 1 1
123 2 2
123 3 3
123 4 4
.................
................
234 1 1
234 2 2
..................
..................

I need first 11 records based on Id1 and Id2. Is there any way to do it in
transformer.

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I'm sure there is, but you should give a better explanation of what 'based on Id1 and Id2' means to get more accurate advice.

This would typically involve stage variables, doing a little 'group change detection' dance and constraining the output to 11 max per group.

:? Actually, that's a Server answer because you posted in the Server forum, but just noticed you marked this as Parallel. Perhaps there's a better approach in a PX job?
-craig

"You can never have too many knives" -- Logan Nine Fingers
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Are you using a Basic Transformer. If yes then Craigs solution would work. Actually it would work even with the Parallel Transformer because all that needs to be done is the change detention and restricting the output to 11.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

If it is PX and if you need first 11n distinct records based on the two field, you have many approach, like using remove duplicate stage in conjunction with transformer, or sort stage with the KeyValueChange option.
But you need to give out more information of what is you exactly need in?
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Post Reply