Change Capture Stage Warning

[vrishabhsagar]

Hi All,

I have two datasets (basically same file), I sort them (set to be hash partitioned on key cols), and then pass them into a Change Data Capture stage. In CDC i mark 2 cols, out of three as key cols, and collect the ouput into another dataset.

When I observe in Director logs, however I see a warning , that reads

" Change_Capture1: When Checking Operator: Defaulting <Col Name> in transfer from beforeRec to outputRec"

where Col_Name is the third col which is NOT marked as key col in CDC.
Am I missing something very obvious?

Also, I am undecided on the best stage [Differnce / CDC] to be used for the following requirement:

"Compare today's dataset with yesterday's dataset; and propogate only those which have been updated or added (compare by key cols)"

Searching on the forums... i found that most of ppl think that "difference" can be used if the dataset is small, for larger datasets, everyone recommends a CDC, is this valid? why/ how?

[ray.wurlod]

The message is only an alert, informing you that the non-key column's default attributes are being mapped to the output link. You don't need to worry about it. If you like, you can demote the message to informational using a local message handler.

[DataDoctor]

I am having same problem. In my company, I can not demote that message to information because when we move the same job to test and prod environment, it will throw same error there. Same time when we call this job from sequencer, we are using "OK" and "Otherwise" option for success and failed links respectively. So if warning is there then it fails the sequencer. Is there any other way to get rid off warning without demoting them to information.

[chulett]

So 'promote' the message handler from the job along with it. They're stored locally in the one of the job specific folders (SC?) with 'msh' in the filename from what I recall.

[ray.wurlod]

Craig has it. Search for Local.msh unless you're using named message handlers, which are stored "globally" in a MessageHandlers directory. They still have the ".msh" suffix, though.