Hi ,
we are using merge stage to identify the insert and updates.its working fine with fewer records.in case of more records it doesnot identifies the inserts ad updates correctly.we are not using any partition technique and running on two nodes.please help me.
Mergestage issue
Moderators: chulett, rschirm, roy
Mergestage issue
thanks n regards
nani
nani
-
- Charter Member
- Posts: 299
- Joined: Wed Nov 13, 2002 5:38 pm
- Location: USA
You are using two nodes. This means you are partitioning on two nodes. This means, in general, half the data goes to one partition and half the data goes to the other partition. If you have not consciously planned for this partitioning then Datastage has done it for you. Erroneously, it would seem. Meaning, you must examine the partitioning scheme used and take a look at the keys you are using to merge the two datasets. Make sure ALL keys from both datasets end up in the same partitions.
A record in one partition will not be merged with a record in the other partition. Keys determine both partitioning and merging. Partitioning and merging keys must be consistent. This may be your problem. To verify that partitioning is your problem, use a single node config file. If the data is correct, then your problem is partitioning.
Or sorting...
A record in one partition will not be merged with a record in the other partition. Keys determine both partitioning and merging. Partitioning and merging keys must be consistent. This may be your problem. To verify that partitioning is your problem, use a single node config file. If the data is correct, then your problem is partitioning.
Or sorting...
-
- Premium Member
- Posts: 892
- Joined: Thu Oct 16, 2003 5:18 am