Issue with Change Capture Stage
Moderators: chulett, rschirm, roy
Issue with Change Capture Stage
I have data coming from Teradata table as Before DataSet.
I have data in a DataSet as After DataSet.
I am trying to do a change capture.
When I run the job for the first time, Before DataSet is empty, and the change_code is 1 and job gives me the records to be inserted.
The issue is, when I rerun the same job immediately without any changes, the change_code is 3 for all the records,indicating a change in the records.
What am I missing here?
I have specified change mode as Explicit Keys -All Values.
There is only 1 key column in my data and the rest of them are value columns.
Thanks in Advance.
I have data in a DataSet as After DataSet.
I am trying to do a change capture.
When I run the job for the first time, Before DataSet is empty, and the change_code is 1 and job gives me the records to be inserted.
The issue is, when I rerun the same job immediately without any changes, the change_code is 3 for all the records,indicating a change in the records.
What am I missing here?
I have specified change mode as Explicit Keys -All Values.
There is only 1 key column in my data and the rest of them are value columns.
Thanks in Advance.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
What are you doing with the records downstream of the Change Capture stage? In particular, are any records loaded into the Teradata table (thereby to become a populated "before" data set)?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Ray,ray.wurlod wrote:What are you doing with the records downstream of the Change Capture stage? In particular, are any records loaded into the Teradata table (thereby to become a populated "before" data set)?
I have a transformer after Change Capture Stage.
From the transformer i 2 links going out.
The first link with constraint "change_code = 1" for new records.
The second link with constraint "change_code = 3" for modified/update records.
I am merging these 2 links using a funnel stage and loading back(doing Upsert) to the same Teradata table.
The issue still remains the same.
Thanks in advance.
Kumar,kumar_s wrote:I guess, its your output dataset points to the After Dataset as input, and hence it might be showing as Copy.
In my change Capture Change stage, i had specified the drop option to True for the copy.
So i should not be getting any records out of the Change Capture stage,
when the records are exact copies.
But still i am getting all the records back , with change_code = 3, indicating change in the records.
The issue still remains the same.
Thanks in Advance
Hi Poorna,
What is the data in After DataSet? Is it the updated output or copy of merged output from funnel?
Do you have any date field from table getting generated at runtime during load or something??So that every time you get updated time, and the data found to be changed?
What is the data in After DataSet? Is it the updated output or copy of merged output from funnel?
Do you have any date field from table getting generated at runtime during load or something??So that every time you get updated time, and the data found to be changed?
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
change capture
There must be some transformation happening after the output from transformer,check for the NULL values and how you are handling in before and after datasets.
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
First isolate the problem. Remove a couple columns at a time and rerun the job until you identify the column or columns that is triggering the change. Examine the contents of that column via a DataStage View editor and, if you can, via a different viewer such as a SQL window. DataStage can add some funny null characters to fields, especially when you change CHAR to VARCHAR.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn