CDC Stage
Moderators: chulett, rschirm, roy
CDC Stage
Hi,
I am trying to compare history file to current file using CDC stage...
The jobs runs fine and could do CDC correctly..but i get some warnings on the other columns which are not keys..and change values
Warning: "CDC: When checking operator: Defaulting "P_PHONE" in transfer from "beforeRec" to "outputRec".
any clue what needs to be done..
Thanks in advance...
I am trying to compare history file to current file using CDC stage...
The jobs runs fine and could do CDC correctly..but i get some warnings on the other columns which are not keys..and change values
Warning: "CDC: When checking operator: Defaulting "P_PHONE" in transfer from "beforeRec" to "outputRec".
any clue what needs to be done..
Thanks in advance...
Kab123,
there are a few who use CDC. Yours would be the first post that I see where no answer at all is forthcoming.
I get the same warning for all columns that are neither key nor used for CDC and think that this is just the way that things work, I can only get use a handler to demote the message to informational.
there are a few who use CDC. Yours would be the first post that I see where no answer at all is forthcoming.
I get the same warning for all columns that are neither key nor used for CDC and think that this is just the way that things work, I can only get use a handler to demote the message to informational.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
exactly..I supressed this message using message handler in dev ...
but this message handler I dont think will get migrated as well with code to production and I dont even have access to add that rule...I dont want to confuse them about this warning..so I thought I would get some suggestions from the forum...
looks like I have to follow the same in PROD too
but this message handler I dont think will get migrated as well with code to production and I dont even have access to add that rule...I dont want to confuse them about this warning..so I thought I would get some suggestions from the forum...
looks like I have to follow the same in PROD too
You could submit this to IBM/Ascential support and see if there is an alternative. We'd be curious to hear what the response it.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Charter Member
- Posts: 166
- Joined: Wed Mar 16, 2005 6:52 am
- Location: Mumbai, India
Hi Kab123,
I think this is why one gets a warning when running CDC with columns other than:keys and values:
* The Target dataset is constructed with columns that are a union of the before and after columns in the dataset. Now there may be columns ini the after dataset that are not present in the Before dataset. There may also be non-key and non-value columns with the same name in both before and after datasets.
What happens in this scenario is that, When the CDC stage marks a row as deleted, the target dataset is populated with the before row.
1. Now columns exclusively present in the after row (which are a part of the union of the two) do not have corresponding before values. These are defaulted to the proper value for the data type of the column(null for nullable; "" for String, 0 for numerics) Hence the "Defaulting" type of warnings.
2. When a non-key and non-value column that is present in both the before and after datasets is encountered, the Before value is considered(row is deleted), the after value cannot be mapped to any target column and there is no way RCP can pass the column forward coz a column with the same name has already been mapped. So the "After" column is dropped. Hence the "Dropping" type of warnings.
Resolution:
Do not pass any non-key or non-value columns to the CDC. instead, use something like this:
It sucks and destroys all performance but there are no warnings.
IHTH
I think this is why one gets a warning when running CDC with columns other than:keys and values:
* The Target dataset is constructed with columns that are a union of the before and after columns in the dataset. Now there may be columns ini the after dataset that are not present in the Before dataset. There may also be non-key and non-value columns with the same name in both before and after datasets.
What happens in this scenario is that, When the CDC stage marks a row as deleted, the target dataset is populated with the before row.
1. Now columns exclusively present in the after row (which are a part of the union of the two) do not have corresponding before values. These are defaulted to the proper value for the data type of the column(null for nullable; "" for String, 0 for numerics) Hence the "Defaulting" type of warnings.
2. When a non-key and non-value column that is present in both the before and after datasets is encountered, the Before value is considered(row is deleted), the after value cannot be mapped to any target column and there is no way RCP can pass the column forward coz a column with the same name has already been mapped. So the "After" column is dropped. Hence the "Dropping" type of warnings.
Resolution:
Do not pass any non-key or non-value columns to the CDC. instead, use something like this:
Code: Select all
After------> Copy-------------------------->Join---|
| (Insert,Change,Copy)| |->
| |--------
---->CDC----->Switch-- Funnel--->
| |--------
| (Delete) | |->
Before-----> Copy-------------------------->Join---|
IHTH
Amey Vaidya<i>
I am rarely happier than when spending an entire day programming my computer to perform automatically a task that it would otherwise take me a good ten seconds to do by hand.</i>
<i>- Douglas Adams</i>
I am rarely happier than when spending an entire day programming my computer to perform automatically a task that it would otherwise take me a good ten seconds to do by hand.</i>
<i>- Douglas Adams</i>
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
I would just ignore the warning and handle it is the message handler. Much easier then trying to split and merge the record down two paths. These warnings happen even if there is no defaulting going on. For example if you drop deletes you still get these warnings, even though the rows with defaulting values are not being written out. Also happens if no deletes are found. So it is largely meaningless unless you have designed your job poorly and try to send delete records down an update or insert path without realising they are there.
To migrate a message handler just copy the file from the MsgHandlers directory in the DataStage home folder to the target server. The details are stored in text files with an msh extension.
To migrate a message handler just copy the file from the MsgHandlers directory in the DataStage home folder to the target server. The details are stored in text files with an msh extension.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
-
- Participant
- Posts: 5
- Joined: Wed Aug 31, 2005 9:19 pm
- Location: Mumbai
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Some of us sleep at night, or take the occasional weekend off! The downside is that now I've got nothing to add to what's already been posted on this thread!kab123 wrote:I guess there are not many people using CDC stage...
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.