Merge stage warning message

[somu_june]

Hi,


Iam using a merge stsage it contains one master link and two update links and one output link and two reject update links. Iam getting a warnings like this . Plz tell me how to solve this


Merge,0: Master record (1) is a duplicate.

Merge,0: Update record (1) of data set 2 was rejected; it was a duplicate record.

Merge,0: Update record (1) of data set 2 is rejected; no are masters left.




Thanks,
Somaraju.

[ray.wurlod]

De-duplicate the inputs. This is a requirement for the Merge stage where there is more than one update input.

Make sure that the master input does not become exhausted while there are still update inputs to be matched (perhaps by re-assessing which is master).

[somu_june]

Hi Ray,


what you mean by De duplicate the inputs. can you explain me in more detail so that it can help me .



Thanks,
somaraju.

[balajisr]

Hi Ray,


what you mean by De duplicate the inputs. can you explain me in more detail so that it can help me .



Thanks,
somaraju.

Deduplicating means your key values should be unique. Make sure your key values are unique for both master and update datasets. You can use remove duplicates stage to deduplicate it.

[khaja]

Hi

Make sure you are following the these things

master and Update rows are merged iff they have the same values in the user specified columns

Unmatched master rows can be kept or dropped
unmatched update rows can be captured in reject link in corresponding output link
and one more important thing is master and update rows should be duplicate free

[kumar_s]

Also look into 'Warn on Reject Updates' and 'Warn on Unmatched Masters'

[ray.wurlod]

While it is true that you can use explicit Remove Duplicates stages, you can also specify de-duplication as part of sorting the input links. Check the "Unique" check box. And this can also be done with a Sort stage.