Datastage 8 bug sort stage ?

Druid_Elf · Post by **Druid_Elf** » Wed Oct 22, 2008 4:05 am

Hi,
I've got the following problem with datastage 8.
I'll first draw the situation a bit...
Oracle stage => Copy stage => Sort stage => Remove duplicate stage => oracle stage

When we ran the job we got the warning :
Remove_Duplicates_21: When checking operator: User inserted sort "Remove_Duplicates_21.SERVICE_Sort" does not fulfill the sort requirements of the downstream operator "Remove_Duplicates_21"

I know that there are a lot of topics about this error and i've read most of them in hoping to find the solution.
But when testing suddenly the error dissapeared. Now it seems if i define in the sort stage how NULL values should be treated (first or last) i get the warning , if I leave this option out then the warning is not shown. Is there any logical explanation for this, or is it indeed a bug ?
These are jobs that are migrated from datastage 7 to 8 ( don't know if this information is vital).
Hope someone can help.

John Smith · Post by **John Smith** » Thu Oct 23, 2008 5:55 pm

Can you clarify how in the Sort stage you define how NULL values should be treated ?

With regards to that message ,it's just a warning and you can ignore it if you like.

Druid_Elf · Post by **Druid_Elf** » Fri Oct 24, 2008 5:49 am

i define in the sort stage to order on the key of the remove duplicate stage and i define NULL shoud be last.

I know it is just a warning, I am just curious as to why datastage throws the warning.

bart12872 · Post by **bart12872** » Fri Oct 24, 2008 6:20 am

Have you exactly the same key in the same order. I mean the same fields AND the same order ?

This warning said much more that you could think.
First, this warning means that your logic of traitement is wrong (your sorts) , but Datastage can correct it. It means that the work is not done correctly or that Datastage is not mastered.

2) This warning means that Datastage correct by itself by doing a second sort before the remove duplicate. So, your sort stage is not necessary because a second sort is done just after. It's not optimal.

3) the fact that is a warning means you don't have the runtime option
"-sort insertion". This option means that Datastage shouldn't add sorts at compilation. So, your performance decrease to not use this option.

that's why you must consider this warning.