one of my job, sometimes run very slowly and some times as expected fast.
it reads a sequential file, remove the duplicate records and insert into oracle target table, using upsert.
will oracle upsert with duplicate source data( if i remove the remove duplicate stage) be faster then my current approach??
Remove duplicate Vs Upsert
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 783
- Joined: Mon Jan 16, 2006 10:17 pm
- Location: Sydney, Australia
Re: Remove duplicate Vs Upsert
Difficult to say with the information you've provided. I would do the remove duplicates instead of relying on the DB engine to update the same row twice.keshav0307 wrote:one of my job, sometimes run very slowly and some times as expected fast.
it reads a sequential file, remove the duplicate records and insert into oracle target table, using upsert.
will oracle upsert with duplicate source data( if i remove the remove duplicate stage) be faster then my current approach??
Wes Dumey
Senior Consultant
Data Warehouse Projects
Senior Consultant
Data Warehouse Projects
I would focus more on the upsert to oracle part to find your fluctuations. Here the performance changes a lot depending on how many records are inserts and how many are updates.
I once did a quick test on our environment where all inserts was 6 times faster then all updates. In most cases the job will be faster if you are able to split the insert and updates and send them to 2 seperate oracle stages.(which are offcource not set to upsert).
I once did a quick test on our environment where all inserts was 6 times faster then all updates. In most cases the job will be faster if you are able to split the insert and updates and send them to 2 seperate oracle stages.(which are offcource not set to upsert).
-
- Premium Member
- Posts: 783
- Joined: Mon Jan 16, 2006 10:17 pm
- Location: Sydney, Australia