Data frm SERVER doesnt match with Parallel in Change Capture

ady · Post by **ady** » Wed Feb 28, 2007 10:37 am

Hi,

I have a server job which I exactly replicated as a parallel job to make the process run faster. I am trying to compare the files from both jobs (Server and Parallel) with change capture stage to make sure the old process data matches with the new process data.

The SERVER job output is a FIXED WIDTH file and the parallel job output is a default SEQUENTIAL FILE.

I am trying to compare these file and have specified all the values as "keys" along with "Explicit keys and Values" option. I have specified the partition type as "HASH" in the change capture stage and selected all the keys in the same order they were mentioned in "CHANGE KEYS" option.

But when I run the job some of the data does not match, i took some columns that are hard coded in both the jobs as keys, these columns dont match when I also specify them as keys to partition the data. When i do not specify them as keys for partitioning they match perfectly.

Can I specify all the "keys" as "keys for partitioning" also ?. Is there another reason why the comparision wont work?

Please help me out. thanks

us1aslam1us · Post by **us1aslam1us** » Wed Feb 28, 2007 10:48 am

Have you specified a sort key there, For both the inputs?
Also make sure that you have specified same key-based partitioning for both inputs.

ady · Post by **ady** » Wed Feb 28, 2007 11:02 am

Yes I have specified the same sort keys. I have specified all the columns mentioned as CHANGE KEYS as sort keys.

they are in the same order.... Is that right?

us1aslam1us · Post by **us1aslam1us** » Wed Feb 28, 2007 12:05 pm

what is the "Director log" say?

Have you tried clearing the preserve partition flag in the Input sequential file stage.

ady · Post by **ady** » Wed Feb 28, 2007 1:59 pm

Director log doesnt say much... It just gives warnings for defaulting values which are not given as change keys or values. Like this:

Change_Capture_72: When checking operator: Defaulting "DOC_TYPE" in transfer from "beforeRec" to "outputRec"

I am trying "clear partition" now, i'll let you know.... thanx

ady · Post by **ady** » Wed Feb 28, 2007 2:13 pm

@us1aslam1us

"Clear Partition" dosent improve the situation but I have another question. I have 1187000 rows in the "before" data and 1188000 rows in the "after" data

Does this effect the change capture process?. It should show the 1187000 rows present in both the files as "copy" and the new rows as edited or inserted. Am i right?

ady · Post by **ady** » Thu Mar 01, 2007 9:18 am

Is "HASH" the right option for sorting, does it effect the data in anyway?

us1aslam1us · Post by **us1aslam1us** » Thu Mar 01, 2007 11:33 am

Yes, there is no issue with hash partitioning. Your warning message suggest that your input and output datatypes are not matching but i would suggest you to go through this:

viewtopic.php?t=99511&highlight=beforerec+outputrec

ady · Post by **ady** » Thu Mar 01, 2007 1:14 pm

@us1aslam1us

Thanks ... that clears my mind about the partitioning.

About the warnings , I did try turning RCP off .. It dosent work, but if I specify all the columns as values or keys ...I shud not get the errors, this happened to me a few times.