Need information change data capture (DS8.7)
Posted: Wed Dec 31, 2014 1:11 am
Dear Team,
We have 150 files . Each files contains 5GB data which is readable by flat file.
Target table contain data about 50 cr.
We are getting full table dump data in the form of flat file (150 files)
We need to update the target table with change records only.
We have develop job as follows:
file ------> change capture ---------> Target table
--------------------|
--------------------|
--------------Ref -Target Table
Question :
1> Is there any stage available so that we can hold the all records in ( Ref -Target Table) memory and pass the file one by one(150 files).
2>Can we concat all files (150) and pass it to the change capture stage so that Ref -Target Table will be read by one time only.(Make sure 150*5=750 GB file read in one way without a bottleneck issue )
3> Suggest good solution for the same.
Thanks
We have 150 files . Each files contains 5GB data which is readable by flat file.
Target table contain data about 50 cr.
We are getting full table dump data in the form of flat file (150 files)
We need to update the target table with change records only.
We have develop job as follows:
file ------> change capture ---------> Target table
--------------------|
--------------------|
--------------Ref -Target Table
Question :
1> Is there any stage available so that we can hold the all records in ( Ref -Target Table) memory and pass the file one by one(150 files).
2>Can we concat all files (150) and pass it to the change capture stage so that Ref -Target Table will be read by one time only.(Make sure 150*5=750 GB file read in one way without a bottleneck issue )
3> Suggest good solution for the same.
Thanks