Data frm SERVER doesnt match with Parallel in Change Capture
Moderators: chulett, rschirm, roy
Data frm SERVER doesnt match with Parallel in Change Capture
Hi,
I have a server job which I exactly replicated as a parallel job to make the process run faster. I am trying to compare the files from both jobs (Server and Parallel) with change capture stage to make sure the old process data matches with the new process data.
The SERVER job output is a FIXED WIDTH file and the parallel job output is a default SEQUENTIAL FILE.
I am trying to compare these file and have specified all the values as "keys" along with "Explicit keys and Values" option. I have specified the partition type as "HASH" in the change capture stage and selected all the keys in the same order they were mentioned in "CHANGE KEYS" option.
But when I run the job some of the data does not match, i took some columns that are hard coded in both the jobs as keys, these columns dont match when I also specify them as keys to partition the data. When i do not specify them as keys for partitioning they match perfectly.
Can I specify all the "keys" as "keys for partitioning" also ?. Is there another reason why the comparision wont work?
Please help me out. thanks
I have a server job which I exactly replicated as a parallel job to make the process run faster. I am trying to compare the files from both jobs (Server and Parallel) with change capture stage to make sure the old process data matches with the new process data.
The SERVER job output is a FIXED WIDTH file and the parallel job output is a default SEQUENTIAL FILE.
I am trying to compare these file and have specified all the values as "keys" along with "Explicit keys and Values" option. I have specified the partition type as "HASH" in the change capture stage and selected all the keys in the same order they were mentioned in "CHANGE KEYS" option.
But when I run the job some of the data does not match, i took some columns that are hard coded in both the jobs as keys, these columns dont match when I also specify them as keys to partition the data. When i do not specify them as keys for partitioning they match perfectly.
Can I specify all the "keys" as "keys for partitioning" also ?. Is there another reason why the comparision wont work?
Please help me out. thanks
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
Director log doesnt say much... It just gives warnings for defaulting values which are not given as change keys or values. Like this:
Change_Capture_72: When checking operator: Defaulting "DOC_TYPE" in transfer from "beforeRec" to "outputRec"
I am trying "clear partition" now, i'll let you know.... thanx
Change_Capture_72: When checking operator: Defaulting "DOC_TYPE" in transfer from "beforeRec" to "outputRec"
I am trying "clear partition" now, i'll let you know.... thanx
@us1aslam1us
"Clear Partition" dosent improve the situation but I have another question. I have 1187000 rows in the "before" data and 1188000 rows in the "after" data
Does this effect the change capture process?. It should show the 1187000 rows present in both the files as "copy" and the new rows as edited or inserted. Am i right?
"Clear Partition" dosent improve the situation but I have another question. I have 1187000 rows in the "before" data and 1188000 rows in the "after" data
Does this effect the change capture process?. It should show the 1187000 rows present in both the files as "copy" and the new rows as edited or inserted. Am i right?
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
Yes, there is no issue with hash partitioning. Your warning message suggest that your input and output datatypes are not matching but i would suggest you to go through this:
viewtopic.php?t=99511&highlight=beforerec+outputrec
viewtopic.php?t=99511&highlight=beforerec+outputrec
I haven't failed, I've found 10,000 ways that don't work.
Thomas Alva Edison(1847-1931)
Thomas Alva Edison(1847-1931)