DSXchange

What I did:
I did a Right outer join on two input files using join stage in to a dataset.
I partitioned both inputs by "Hash" on key using inlink partition.

Observation:
When I sort on keys in Ascending order, the stage works fine.
When I sort on keys in Descending order, the stage doesn't give the expected result and also throws the following warning.

Warning:
Join: When checking operator: User inserted sort "Join.Left_Link_Sort" does not fulfill the sort requirements of the downstream operator "APT_JoinSubOperator in Join"

Question:
Is descending order not applicable to "Join" stage.
I tried and couldn't get a solution from Online manuals and a search on this Forum.

Thanks in advance,
Lakshman.

Have you sorted both the links in same order ?

tagnihotri wrote:Have you sorted both the links in same order ?

Yep.

There's no reason descending order should not work, provided that you really have specified partitioning and sorting exactly the same for both links, or Entire on the right input (which carries almost no penalty in an SMP environment).

What do you mean by "does not work"? Are the appropriate columns nullable, since they may receive NULL as the result of the outer join?

ray.wurlod wrote:There's no reason descending order should not work, provided that you really have specified partitioning and sorting exactly the same for both links, or Entire on the right input (which carries almost ...

Thanks a lot. I checked again. I have specified partitioning and sorting exactly the same way. Also now i tried with "Entire" partition for the "Right" input, Still it doesn't.

Kindly update me if i need to give any further information to make this clear.

sorted in descending order + then joined ryt

check the join component sort options .it should work...
please let me know ..

lakshmEn@gmail.com wrote:What I did:
I did a Right outer join on two input files using join stage in to a dataset.
I partitioned both inputs by "Hash" on key using inlink partition.

Observation:
When I sort on keys in Ascending order, the stage works fine.
When I sort on keys in Descending order, the stage doesn't give the expected result and also throws the following warning.

Warning:
Join: When checking operator: User inserted sort "Join.Left_Link_Sort" does not fulfill the sort requirements of the downstream operator "APT_JoinSubOperator in Join"

Question:
Is descending order not applicable to "Join" stage.
I tried and couldn't get a solution from Online manuals and a search on this Forum.

Thanks in advance,
Lakshman.

I introduced "Sort stage" before my "Join" stage.

Now my "Join" stage has no inlink sort. It just does "Auto" partition.

In the "Sort" stage, if I sort in ascending order, it works.
In the "Sort" stage, if I sort in descending order, it doesn't.

Have you provided a reproducible example to your support provider? If so, what feedback did you get?

Now i introduced "Peek" stage like below to the job which actually was working wrong with sort order as descending.

Left_Input_file -> Sort stage -> Peek ->
Join -> Output
Right_Input_file -> Sort stage -> Peek ->

To my surprise, it ran fine and gave the expected result.

Now i removed the peek and still surprised to see that it works fine.

My Objective got resolved but doubt still remains. What could have happened by the mere introduction on "Peek"??? If any body can get a clue kindly update me.

ray.wurlod wrote:

Code: Select all

select * from users where clue > 0;

0 rows returned.

:D

Thanks everybody for the help.

Is there any thing i need to do to close this posting?

There should be a resolve button at the top. Just press that to mark it resolved.