Rgrding Config files usage in DS 8x PX

[srini.dw]

Hi,

I tried to run a job using a 2- node config file.
The job has an 2 input datsets being joined (INNER JOIN) to 1 teradata table and the output being sent to a teradata table (target). There are no records dropped in this process and data gets successfully loaded into the target DB table.

Note: The 2 input datasets are also created with the same 2-node config file.

But when i run the same using 4-node config file, the records get dropped in the joiner stage resulting into partial data getting into a the target DB table.

Note: The 2 input datasets are also created with the same 4-node config file.The join columns are hash partitioned and sorted before the joining occurs.

Please do let me know what can be done in this case.

[ArndW]

Have you done any repartitioning of the data in any of the stages leading up to the join stage? Note that you might be doing implicit repartitioning; and some stages change the output sort order and that might affect a 4 node but not a 2 node hashing partitioning.

[Jai_sahaj]

Check the sequence of fields for partitioning and sort on each input link to the join.