Hi,
I'm using change capture in my job. I'm testing with a small set of data.
I'm not geting the desired result when i try to run change capture stage in parallel mode, but it works fine and gives the desired result when run in sequential mode, and this doesn't solve my purpose as i'm going to run the job with a very large set of data.
Using Change Capture Stage
Moderators: chulett, rschirm, roy
Using Change Capture Stage
Om Prakash
"There are things that are known, and there are things that are unknown, and in between there are doors"
"There are things that are known, and there are things that are unknown, and in between there are doors"
Change capture stage is build to run on parallel execution mode. It shouldnt give you any undesired result. But you could mention what is that you desire to get and what is the output from the stage. Perhaps you could concentrate on Partiton method used.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Kumar,
You are right , its working when i hash parton both the input set....but am i doing the right thing????
You are right , its working when i hash parton both the input set....but am i doing the right thing????
![Question :?:](./images/smilies/icon_question.gif)
kumar_s wrote:Change capture stage is build to run on parallel execution mode. It shouldnt give you any undesired result. But you could mention what is that you desire to get and what is the output from the stage. Perhaps you could concentrate on Partiton method used.
Om Prakash
"There are things that are known, and there are things that are unknown, and in between there are doors"
"There are things that are known, and there are things that are unknown, and in between there are doors"
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
It is vital that both Data Sets are identically partitioned on the comparison keys, so that valid comparisons will be performed. It is highly desirable that both Data Sets are identically sorted on the comparison keys, so that efficient use can be made of memory. What do you mean by "the desired result"? And what are you getting versus what the Change Capture stage is documented as generating?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Yes you are doing the right thing.opdas wrote: its working when i hash parton both the input set....but am i doing the right thing????![]()
As mentioned, both the dataset need to be sorted and corectly partitioned across nodes to get the correct result.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'