Hash File creation taking long time

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
saikrishna
Participant
Posts: 158
Joined: Tue Mar 15, 2005 3:16 am

Hash File creation taking long time

Post by saikrishna »

Hi

I have the following job

DB Stage-> Tfm ->Aggregator ->Hash File

From DB Stage, we are extracting 3 million records and doing aggregation which is converting into 2.3 million records and then loading to hash file.

This job is taking around 55 minutes to load.

If I replace hash file with sequential file in above job, it is running in 11 minutes.

What could be the problem taking this long time. How to improve the performance of hash file creation job.

Settings of Hash File:
No check box is selected among the below options:
i. Allow Stage write cache
ii. create file
iii. clear file before writing
iv. backup existing file.

Settings at administrator level:
Tunables ->Hash file size
Read cache size->128MB
write cache size-> 128 MB


Please let us know your suggestions to improve the performance of the job.

Thanks
Sai
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

This can turn into a large conversation, but you need to precreate the hashed file with an appropriate size for the volume by setting the Minimum Modulos to something other than the default of 1.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply