HI All,
Please let me know how to bring down the timing for the job run . To load the data it is taking 4 hr . There are 3 lookup table in the mapping which is haivng 20 lacs of record .The source of the mapping is having 8lacs . There are 8lookups.I have used link partitioner and collector .I have set the properties of this partioner to 'Hash' and collector as 'sort/merge' .the buffer size i have incresed to 512 and timout to 500.
Please suggest how to improve the performance using this server edition.
Thanks
rubu
How to Tuning of the server JOB which is taking 4hrs ?
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 34
- Joined: Thu Jul 29, 2004 1:03 pm
Re: How to Tuning of the server JOB which is taking 4hrs ?
Hi Rubu
You say that you have three lookups, that have 20lacs records in each??
In this case this is the beginning of your problems. A lookup is not meant to be used against 20 lacs records.
You should do better joning the source to your lookups in SQL or you can use the mergr stage.
Also it will benefit to set the Inter processing buffering on
Thanks
You say that you have three lookups, that have 20lacs records in each??
In this case this is the beginning of your problems. A lookup is not meant to be used against 20 lacs records.
You should do better joning the source to your lookups in SQL or you can use the mergr stage.
Also it will benefit to set the Inter processing buffering on
Thanks
Rubu wrote:HI All,
Please let me know how to bring down the timing for the job run . To load the data it is taking 4 hr . There are 3 lookup table in the mapping which is haivng 20 lacs of record .The source of the mapping is having 8lacs . There are 8lookups.I have used link partitioner and collector .I have set the properties of this partioner to 'Hash' and collector as 'sort/merge' .the buffer size i have incresed to 512 and timout to 500.
Please suggest how to improve the performance using this server edition.
Thanks
rubu
-
- Participant
- Posts: 12
- Joined: Tue Dec 21, 2004 7:57 am
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom