Capturing record count for scd2 Insert and Update
Posted: Fri Jan 30, 2015 1:58 am
Hi All,
This is a scd2 implementation job and I want capturing number of records inserted and updated to my target. We have two separate links to Oracle connecter stage (one for insert and one for update).
In the current audit capturing process we are reading the count from stage_name.link_name from the output link of CDC stage (this is our source count) and same process from insert and update link to the target connector stage (addition of this two will give us the target count)
This process will work fine if all my records from source are new. So all record will go in the insert link and my source count and target count will match.
If there is any updated records then for that given updated record we will have a 1 record in insert link (new record) and one in the update link (to expire the old record). SO there will be 2 counts for 1 updated record, which will create a difference in count in the audit table.
The audit job is a separate job.
Our after job subroutine (in SCD2 load job) is creating a XML report of the job log and our audit job processes the xml file to extract the information using stage_name.link_name.
Is there any changes I need to do in my job so that I can capture the correct count for a updated record.
Regards
~Manoj
This is a scd2 implementation job and I want capturing number of records inserted and updated to my target. We have two separate links to Oracle connecter stage (one for insert and one for update).
In the current audit capturing process we are reading the count from stage_name.link_name from the output link of CDC stage (this is our source count) and same process from insert and update link to the target connector stage (addition of this two will give us the target count)
This process will work fine if all my records from source are new. So all record will go in the insert link and my source count and target count will match.
If there is any updated records then for that given updated record we will have a 1 record in insert link (new record) and one in the update link (to expire the old record). SO there will be 2 counts for 1 updated record, which will create a difference in count in the audit table.
The audit job is a separate job.
Our after job subroutine (in SCD2 load job) is creating a XML report of the job log and our audit job processes the xml file to extract the information using stage_name.link_name.
Is there any changes I need to do in my job so that I can capture the correct count for a updated record.
Regards
~Manoj