Problem in Deduping

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

parag - I was just passing on the information to Satheesh, that poster was aware of the fact that the input file is delimeted by '}' and was using the same for this purpose.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

This is all fine and dandy. However, the answer is clear - there cannot repeat can not be duplicate key values in a hashed file. Period. End of story.

In spite of what has been posted as to how the records may look the same to the naked eye, there must be differences between them. Period. End of story. Sensing a pattern here? :wink:

Rather than assuming there 'can not be extra spaces' or things of that ilk, the OP needs to look much closer at their source data and the design of the hashed file. Something in there is not at they are expecting and is causing these apparent duplicates. This is not something we can solve for them, other than to point them in the right direction.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply