Page 1 of 1

Issue with Join Stage

Posted: Fri Jul 15, 2011 2:32 pm
by chkumar.ds
I have a an issue with JOIN (Inner Join ) stage ,in my case i have 2 files where i use inner join and get the output for the matching records ...(did hash partition on the join keys ) .
in my case when i run the job :

in seqential mode it is giving me one output
in parallel mode (2node) it is giving me one output
in parallel mode ( 4node) it is giving me one output
in parallel mode (8 node) it is giving me one output ...

my question is does join stage generates different outputs for diiferent node config file's of the same design jobb.......

Please help me with the issue ...and i really appreciate your help and inputs ...

Posted: Fri Jul 15, 2011 5:50 pm
by ray.wurlod
It looks to me like there's only one combination of keys that match. Any other nodes will be idle for this link.

Re: Issue with Join Stage

Posted: Fri Jul 15, 2011 8:23 pm
by shivadrash
Hashing alone wont do. Sort them on the joining Keys,