Hi Everyone,
I have a job design which pulls data from source compares with old data using change capture stage and then performs inserts/updates/deletes.
Design:
Oracle EE stage ( OLD DATA )
|
Source (Oracle EE stage)---> CC ---- updates/inserts/deletes.(Oracle EE stage)
Apart from these stages job has 3 join stages and one transformer.
Now data has grown huge and it's taking more than 2 hours to compare 6-8 million records and some additional time for inserts/updates/deletes.
Source doesn't have any time stamp fields to pull only delta records. So i am doing full image comparison.
Could anyone suggest how to increase the performance.
Any inputs would really appreciated.
Thanks in Advance
Improve Performance of a job
Moderators: chulett, rschirm, roy
Re: Improve Performance of a job
1. You can create a checksum to compare.
2. You need to split the job.
DS User
2. You need to split the job.
DS User