Page 1 of 1

Merge stage

Posted: Fri Mar 08, 2013 7:55 am
by lathalr
Hi All,

I have two different files. I want to merge two files but I don't have any common keys. So I have created a dummy key to merge two files. Master link having 13rows and reference having 5rows. But after merging, I am getting only 17rows. One row is missing. Please help me to get 18rows in my output.

Thanks in advance.

Posted: Fri Mar 08, 2013 9:06 am
by prasannakumarkk
In the both the link choose the partition method as hash and provide dummy column created as key

Posted: Fri Mar 08, 2013 9:25 am
by prasson_ibm
Hi,
Take a reject link from update and check "drop" method in merge stage, lets see one record should go to reject.

Posted: Fri Mar 08, 2013 10:57 am
by sudha03_vpr
Is the key unique on both the links ?

Posted: Fri Mar 08, 2013 11:26 am
by prasson_ibm
Hi,
I think master inputs of the merge stage should not be duplicate on the key column.as part of preprocessing your data for merge stage,you should remove duplicate records from master dataset and if you have more then one update datasets,you must remove duplicate records.

Posted: Fri Mar 08, 2013 11:31 am
by sudha03_vpr
Yes you are right prasson. If its duplicate then the data rejects are obvious.

Posted: Fri Mar 08, 2013 12:05 pm
by priyadarshikunal
Why really you need to merge when you do not know the keys? Anyways if you are merging it based on keys, and expecting total number of records, which means there won't be any record which will match. Why not use funnel in that case. May be just a bad choice of stage or you don't know why you need to merge it.

A little more details will help here.