DSXchange

Im using 6 datasets using Left Join stage. 3rd dataset is not joining properly. Join works fine if i use the Hash partitioning on 3rd dataset and mainstream data.

But then should i have to revert back hash partitioning to Automatic so that output can join properly to other remaining datasets. Please suggest.

As a pre-requisite to JOIN stage, the dataset data should be pre- hash partitioned and sorted on the JOIN key columns before they are fed to the JOIN stage.

If the data in the dataset is hash partitioned and sorted on JOIN keys in the previous job, then you can retain the same partitioning by setting Partitioning to "SAME" instead of "AUTO".

Hope this helps.

As a pre-requisite to JOIN stage, the dataset data should be pre- hash partitioned and sorted on the JOIN key columns before they are fed to the JOIN stage.

If the data in the dataset is hash partitioned and sorted on JOIN keys in the previous job, then you can retain the same partitioning by setting Partitioning to "SAME" instead of "AUTO".

Hope this helps.

Dump the score to learn what partitioning Auto actually gives you.

just check whether 'APT_NO_PART_INSERTION' and 'APT_NO_SORT_INSERTION' env variables are disabled or not. These two variables decide whether auto partitioning and auto sorting should occur or not.

Thanks,
Nripendra

A clarification, in view of the thread subject. The second of these environment variables governs whether tsort operators will be inserted, and has nothing at all to do with partitioning.