CDC Stage Issue...

[ds_search2008]

Hello,

Has anyone experienced the following problem and have found a possible solution to it?

In one of my DS jobs I am passing 25 columns from Sort to CDC stage. While doing so all the 24 columns are getting into the CDC stage and only one column is found missing. I have checked the Output--> Columns and Mapping tabs within the Sort stages in before and after paths. I'm able to see the missing column.

Code: Select all

                                DB2 Stage (Before run)
                                 |
                                Sort
                                 | 
Seq File----Transformer---Sort---CDC ...
(After Run)

Also, both before and after sort stages, have the similar meta data definitions.

In CDC stage

1. I have set the Options-->Change Mode as "Explicit Keys, All values".
2. This column is available in the Input tab.

However, within the Column Selection window one of the columns from the previous stage is missing.

Initially I looked into the column propagation within columns tab and searched this forum. I don't find a solution for this scenario. (Not sure if I need to use a different key word for search.. ) Thought of seeking your help.

Kind Regards and Many Thanks in advance

[priyadarshikunal]

Also, both before and after sort stages, have the similar meta data definitions.

similar metadata or same.

If its only one column you can identify whats different.

[samsuf2002]

Make sure the columns you are comparing in CDC are same in both input links.

[mandyli]

Hi

use same meta data before and after and also CDC input and output tabs.

Or else simple way try to write into flat file before CDC stage for trace the missing columns.

Thanks
Man

[ds_search2008]

Thank you all for your valuable suggestions and time.

priyadarshikunal

similar metadata or same.

Yes, both before and after metadata are same. Meaning, column name, data type, length are all one and the same.

As per mandyli's suggestion, I tried writing the before and after path into different sequential files. I'm able to capture the missing column. However, this one column is not found in CDC stage.

The log is showing the following fatal error:

CDCStage: Error when checking operator: On input data set 0: Could not find input field "Account_Num".

Kindly help me on fixing this error.

Many Thanks

[priyadarshikunal]

What about RCP?

Are you actually getting the column in input links. try putting all the columns to sequential file or peek to validate which columns are present in links.

[ds_search2008]

I tried different possibilities and finally resolved this issue. Now the job is working fine without any warning message.

Practical steps that I followed: (may be useful for anybody who is facing similar warning in future)

• Step:1 I deleted the missing column (Account_Num) from all the stages from start till end.
  Step:2 Defined the Key columns in before path, after path and CDC stage consistently.
  Step:3 Rechecked and fixed the Partition type in sort and CDC stages.
  Step:4 Rightly fixed the "Preserve Partitioning" options appropriately from start till end.
  Step:5 Tried to load the target table. The table was loaded properly.
  Final Step: Inserted the missing column manually throughout the flow.

I was able to suppress this warning message

Code: Select all

CDCStage: Error when checking operator: On input data set 0: Could not find input field "Account_Num". 

Again thank to everybody for your help and time.