Page 1 of 1

Posted: Wed Nov 03, 2010 1:49 pm
by mhester
Change your "same" partitioning to partition by the key you are using. Also, change the setting for "Log Statistics" to True so you can see in the director what the stage sees as inserts or edits.

Could be that the files are truly not the same. From Unix do an md5sum on both just to be sure.

What kind of configuration file are you using?

Posted: Thu Nov 04, 2010 8:00 am
by srinivas Guduru
mhester wrote:Change your "same" partitioning to partition by the key you are using. Also, change the setting for "Log Statistics" to True so you can see in the director what the stage sees as inserts or edits.

Could be that the files are truly not the same. From Unix do an md5sum on both just to be sure.

What kind of configuration file are you using?
Now I used "Auto" and "Hash" partions but I didnot find any difference in output.

Change Mode = ExplicitKeys AllValues

What my expectation is output of ChangeCpture stage should be zero byte file
Can you please look into this and advise me

Thanks
Srini

Posted: Thu Nov 04, 2010 8:49 am
by mhester
Have you verified in Unix that the 2 files are the same?

Here are some things that I try when stuff like this happens -
  • Change to a single node configuration (take partitioning out of the equation)
    Ensure that column types are the same that are being compared
    If using seq files and the expected results would be no output based on the properties defined, then I make sure the files are identical.
Turning on "Log Statistics" is really helpful and you should use it.

Posted: Thu Nov 04, 2010 9:55 am
by ray.wurlod
The CDC stage is particularly for working with data captured from transaction logs by IBM's Change Data Capture product (formerly Data Mirror). Is this the purpose to which you are putting the stage? If not, investigate the Change Capture, Difference or Compare stages.