Data sorting question

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Nicole
Premium Member
Premium Member
Posts: 93
Joined: Mon Oct 22, 2007 12:55 pm
Location: Albany
Contact:

Data sorting question

Post by Nicole »

Hellooo. Below is the the data I am looking at. I need to keep the row with the highest value in Col4 for each group of data. What is the best way to do this? When I use the sort descending and then remove dups and keep the first row, it sorts all of the numbers so that in Col4 it would be all of the highest numbers together, not grouped. Am I making sense? I know this is possible but it's just not coming to me that easily. Any advice is appreciated! :) Thanks, Nicole.

Col1 Col2 Col3 Col4
12345 01 23 2
12345 01 23 1
12345 01 23 3

12345 01 29 2
12345 01 29 1
12345 01 29 3

23456 00 24 1
23456 00 24 2
23456 00 24 5
23456 00 24 6
23456 00 24 4
23456 00 24 3

23456 00 05 2
23456 00 05 1
23456 00 05 3
23456 00 05 5
23456 00 05 4
23456 00 05 6
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Assuming you posted your incoming data, can you post what you would want the output to look like for that sample, please?
-craig

"You can never have too many knives" -- Logan Nine Fingers
Nicole
Premium Member
Premium Member
Posts: 93
Joined: Mon Oct 22, 2007 12:55 pm
Location: Albany
Contact:

Post by Nicole »

No problem - below is what I would want the output to look like. Basically I want to keep the row with the highest number in COL4 for each group of data. Does that help? I was trying to sort and remove duplicates, but it just wasn't working, at least not the way I did it!

Col1 Col2 Col3 Col4

12345 01 23 3

12345 01 29 3

23456 00 24 6

23456 00 05 6
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Did you try creating a Key Change Column when you sorted descending? It will be a 1 for the first record in each key group and a 0 otherwise, you'd want all the 1s.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply