Removing Duplicates

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
arnie_nits
Participant
Posts: 101
Joined: Mon Aug 13, 2007 2:32 am

Removing Duplicates

Post by arnie_nits »

Hi,

I want to get rid of any number of duplicates.

example.

111,111,111,111

should be
111,91110,91111,91112

There can be any number of duplicates.

Please suggest.

Regards,
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Is this in one field, i.e. in one string and duplicates are defined by commas?
arnie_nits
Participant
Posts: 101
Joined: Mon Aug 13, 2007 2:32 am

Post by arnie_nits »

Hi,

This is in one field and no commas.

it comes in rows.
karthi_gana
Premium Member
Premium Member
Posts: 729
Joined: Tue Apr 28, 2009 10:49 pm

Post by karthi_gana »

can you post your requirement clearly?
Karthik
arnie_nits
Participant
Posts: 101
Joined: Mon Aug 13, 2007 2:32 am

Post by arnie_nits »

id
111
111
111
111

required result
id
111
9110
9111
9112
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

That doesn't really explain much. :?

Can you spell out your requirements in words? It seems you need to take duplicate values and prefix them with a '9' and then... ???
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Do it with stage variables in a Transformer stage. Detect that the value is the same as in the previous row. If so, add prefix and suffix, and increment the stage variable that contains the suffix.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply