Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.
Moderators: chulett , rschirm , roy
zulfi123786
Premium Member
Posts: 730 Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore
Post
by zulfi123786 » Mon Sep 28, 2009 4:24 am
the default size is 128 mb, how can i change this setting to a higher value for better performance?
ArndW
Participant
Posts: 16318 Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:
Post
by ArndW » Mon Sep 28, 2009 5:02 am
Chances are very, very high that your job's bottleneck is not in reading a hashed file but somewhere else, thus changing buffer and cache sizes, if possible, would not make a difference.
Are you asking about reference reads, or perhaps writes? Also, perhaps you might be looking at public/private link caching - which one might it be?
chulett
Charter Member
Posts: 43085 Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO
Post
by chulett » Mon Sep 28, 2009 7:01 am
Are you asking how can you change it or how can you know what to change it to? And are you thinking about changing the
default?
Be very careful there, it can have quite unexpected consequences.
-craig
"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607 Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:
Post
by ray.wurlod » Mon Sep 28, 2009 7:16 am
What makes you believe that larger buffer sizes relate to "performance" (whatever that is) being improved?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
zulfi123786
Premium Member
Posts: 730 Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore
Post
by zulfi123786 » Mon Sep 28, 2009 7:31 am
I am doing a look up on a hashfile and in log i can see that it is Private caching.....
What is the difference between Private caching and public caching???
ArndW
Participant
Posts: 16318 Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:
Post
by ArndW » Mon Sep 28, 2009 7:34 am
There is a whole PDF documenting the differences and link caching functionality, but in most cases it is not necessary to turn on this functionality. Just load your hashed lookup file to memory (just those columns you intend on using) and DataStage will, in almost all cases, do a quick and good job for you.
zulfi123786
Premium Member
Posts: 730 Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore
Post
by zulfi123786 » Tue Apr 06, 2010 3:39 am
ArndW wrote: There is a whole PDF documenting the differences and link caching functionality.
In which datastage document is it discussed? I cant find anything in the server job developer guide
ray.wurlod
Participant
Posts: 54607 Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:
Post
by ray.wurlod » Tue Apr 06, 2010 3:49 am
dsdskche.pdf
I can't remember what the title of the manual is.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Posts: 43085 Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO
Post
by chulett » Tue Apr 06, 2010 5:26 am
Technical Bulletin, Part No. 00D-TB003, Hash Stage Disk Caching
-craig
"You can never have too many knives" -- Logan Nine Fingers