Hi,
I have a Job where I am reading Data From the Dataset, and a Copy Stage sends data across 3 separate Links with sort stage followed by a dataset. Input data has 20 columns and for processing 20 rows the Job is taking almost 3-4 minutes.
It seems to be issue with Copy Stage. When I observed through a Monitor I could find bottleneck at Copy Stage.
The performance changes with change in the input partition for Copy stage. ( I could get best performance of 2.5 min through HASH partition choosing random keys , as I dont have primary key for input data)
Any idea/solution for this Problem.
Performance issues with the Copy Stage
Moderators: chulett, rschirm, roy
-
Inquisitive
- Charter Member
- Posts: 88
- Joined: Tue Jan 13, 2004 3:07 pm
Hi,
I think copy stage only givening good performance b'cus w r also used copy stage for lookup process. we r getting data from Shared container then using copy stage sending data to Lookup stage for lookup it will work fine. i hope u don't change any default setup in the copy stage.
like
[color=blue][size=12][ Shared container ----> Copy tage -----> lookup [/size][/color]]
I think copy stage only givening good performance b'cus w r also used copy stage for lookup process. we r getting data from Shared container then using copy stage sending data to Lookup stage for lookup it will work fine. i hope u don't change any default setup in the copy stage.
like
[color=blue][size=12][ Shared container ----> Copy tage -----> lookup [/size][/color]]