Sort stage warnings

[kollurianu]

Hi All,

I am getting following warnings in sort stage,

sort_time_diff: Data claims to already be sorted on the specified keys the 'sorted' option can be used to confirm this. Data will be resorted as necessary. Performance may improve if this sort is removed from the flow

Any thoughts why I am getting above warnings can some shed light?

All I am trying to do is sorting that stage and in the next stage remove duplicates.

appreciate your help in advance.

Thank you all.

[divesh]

Take out the sort stage and do the hash partition and perform the presort on the Input tab in remove duplicate stage.

[kollurianu]

I did that still remove duplicates is giving a warning as below

rmv_dup: When checking operator: User inserted sort "rmv_dup.lnk_to_rem_dup_Sort" does not fulfill the sort requirements of the downstream operator "rmv_dup"

[kwwilliams]

Two things. Its appears that you are sorting your data multiple times, that is why you are getting this warning:

sort_time_diff: Data claims to already be sorted on the specified keys the 'sorted' option can be used to confirm this. Data will be resorted as necessary. Performance may improve if this sort is removed from the flow

It is saying you've already done this once before, you don't need to do this again. If you would like you can change the sort method to "don't sort previously sorted".

On your remove duplicate stage, your key columns must be listed int eh same order in the remove duplicate stage as they are in the sort stage (or link) and the options for the sort and remove duplicates need to be the same (Case Sensitive and EBCDIC). If the order or options are different this warning will appear because the sort and remove duplicate options are based upon two different algorithims.