Hello All,
I want to know how difference stage will work. i want to implement this stage for delta loads.
I need to create a delta file based on based on current day file and previous day file. I want only new records in the delta file.
Thanks in Advance
Arvind
difference stage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Hi Arvind,
I would advise you to search this forum for CDC stage. The CDC stage is the best stage for delta loads.
The CDC stage has 4 options for Drop Out and in your case you can define Drop Out for Insert = False and Drop Out for Delete/Edit/Copy = True. The delta file will have only new records.
HTH
--Rich
I would advise you to search this forum for CDC stage. The CDC stage is the best stage for delta loads.
The CDC stage has 4 options for Drop Out and in your case you can define Drop Out for Insert = False and Drop Out for Delete/Edit/Copy = True. The delta file will have only new records.
HTH
--Rich
Thanks Rich and Ray
Hi Rich,
I looked into the CDC stage and I think it is the solution for my job.
Please let me know how to get the change records
1) when i have a single column as a key?
2) when i have complete record as the key ( is there will be any constrain on the column length).
Thanks in advance
Arvind
Hi Rich,
I looked into the CDC stage and I think it is the solution for my job.
Please let me know how to get the change records
1) when i have a single column as a key?
2) when i have complete record as the key ( is there will be any constrain on the column length).
Thanks in advance
Arvind
Hi Arvind,
In the CDC stage there is an option known as Change Mode.
1. If you are having a single column for detecting change and if it is a key use Explicit Keys and All Values option in Change Mode. In the Change Keys section add the column.
2. Column Length should never be a constraint. Just give it a try and if you are facing problems come back and post it.
3. Link ordering is another option you should take care when using the CDC. The link which represents the Current Day records should be the After Link
4. Finally, use the search option in this forum or Read Chapter 31 in Parallel Job Developers Guide.
HTH
--Rich
In the CDC stage there is an option known as Change Mode.
1. If you are having a single column for detecting change and if it is a key use Explicit Keys and All Values option in Change Mode. In the Change Keys section add the column.
2. Column Length should never be a constraint. Just give it a try and if you are facing problems come back and post it.
3. Link ordering is another option you should take care when using the CDC. The link which represents the Current Day records should be the After Link
4. Finally, use the search option in this forum or Read Chapter 31 in Parallel Job Developers Guide.
HTH
--Rich
Chage capture Stage is throwing the warnings
Hi All,
I have created the job to create a delta file. job is running and show finished but throwing the warnings, giving the wrong data in the output.
I used two ftp stages change capture stage and a dataset.
Change_Capture_7: When checking operator: Defaulting "TICKER6" in transfer from "beforeRec" to "outputRec".
Throwing above warning for all columns.
Please let me know what i'm doing wrong.
Thanks in Advance
Arvind
I have created the job to create a delta file. job is running and show finished but throwing the warnings, giving the wrong data in the output.
I used two ftp stages change capture stage and a dataset.
Change_Capture_7: When checking operator: Defaulting "TICKER6" in transfer from "beforeRec" to "outputRec".
Throwing above warning for all columns.
Please let me know what i'm doing wrong.
Thanks in Advance
Arvind