Dear DS Guru's ,
I have an issue in production which one of the jobs taking 13hr to write the data into hash file .
Job design :extract the data from Teradata stage and transformer(droping one column out of 4 columns ) and writing 3 columns into hash file .
write mode :clear file before writing
Teradata pulling data within 15 min then writing the hash file is taking more than 13hr .
Record count :814375543
Please help me to tune this job .
Your timely help is very much appreciated .
Regards,
Mano
Write to hash file is taking long time Nearly 13hr
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 30
- Joined: Fri Sep 26, 2008 9:33 pm
- Location: Bangalore
Re: Write to hash file is taking long time Nearly 13hr
Which unix flavour is that?
Thanks
Ram
----------------------------------
Revealing your ignorance is fine, because you get a chance to learn.
Ram
----------------------------------
Revealing your ignorance is fine, because you get a chance to learn.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
What's the key? How big (wide) are the rows? What hashing algorithm have you specified? What size for the hashed file? Get these right and you will be able to populate it faster.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 30
- Joined: Fri Sep 26, 2008 9:33 pm
- Location: Bangalore
Or to choose a starting size. So you start off with the smallest possible hashed file and then proceed to write 814 million records to it. It spends the majority of your 13 hours basically resizing and expanding the hashed file.manu.dwhds wrote:Create a file --Not selected hence options disabled to chosse hash method.
I'd start by suggesting you find the Hashed File Calculator utility that I assume is still part of the delivery media. Answer a few questions about your hashed file and it will give you values to plug into those 'Create File' options you currently have disabled so that it can be created at a proper size for the volume of data it will be holding.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers