In the latest DataStage Installation Guide there are 2 new kernel parameter configuration recommended, the MAXUPROC and NOFILES. For Solaris the following values are reccommended
MAXUPROC - Minimum Value of 100 per CPU
NOFILES - Minimum Value 1000
The upgrade/install guide does not state the purpose of these kernel parameters for DataStage. I have searched through SUN documentation and found that the MAXUPROC is the limit of no of allowed process per user but I cant seem to find out the NOFILES parameter.
Have you some information on these kernel parameters that might come in handy? What happens if they are not specified, will the upper value be unlimited or...?
NOFILES and MAXUPROC Kernel Parameters in DS 7.5
Moderators: chulett, rschirm, roy
Hi,
theese are OS kernel parameters that you need to consult with sys admins to reconfigure.
they will also mention that the NOFILES is the limit of simultaniously open files (I can't recall right now if its a total limit or per process)
in future cases like this consult your sys admins for OS stuff to get a quick answer (if your not yet familiar with theese persons get to be their friend asap )
IHTH,
theese are OS kernel parameters that you need to consult with sys admins to reconfigure.
they will also mention that the NOFILES is the limit of simultaniously open files (I can't recall right now if its a total limit or per process)
in future cases like this consult your sys admins for OS stuff to get a quick answer (if your not yet familiar with theese persons get to be their friend asap )
IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
MFILES is the number of files which Universe can have open within one process. This has to live under the limit at the UNIX level. Some UNIX platforms do not have a limit. NOFILES is the UNIX kernel parameter which sets the limit at the UNIX level. If MFILES allows a process to exceed the UNIX level then it will core dump. So you have to set these together. Most UNIXs have this limit. I think AIX may be the only one which does not. I am doing all this from memory. I would have to look it up to be sure. The UNIX command sysdef will usually tell what the UNIX kernel parameters are set all.
The uvconfig file sets these for Universe. You have to do a uvregen to make these parameters permanent. At TCL in the UV account in Universe or the DataStage engine type in CONFIG ALL to see these parameter settings.
Do a search. I think Ray and Ken both have talked about these parameters and the rotating file pool.
All this is covered in both the DataStage and Universe manuals. I explained above how files get closed when a Universe process gets close to the MFILES limit. You can count how many files are opened or need to be opened in a job by counting the links. If a file gets opened and closed a lot then your job runs slower. Your rows per second go down especially in processing with hash files.
The uvconfig file sets these for Universe. You have to do a uvregen to make these parameters permanent. At TCL in the UV account in Universe or the DataStage engine type in CONFIG ALL to see these parameter settings.
Do a search. I think Ray and Ken both have talked about these parameters and the rotating file pool.
All this is covered in both the DataStage and Universe manuals. I explained above how files get closed when a Universe process gets close to the MFILES limit. You can count how many files are opened or need to be opened in a job by counting the links. If a file gets opened and closed a lot then your job runs slower. Your rows per second go down especially in processing with hash files.
Mamu Kim
-
- Charter Member
- Posts: 13
- Joined: Tue Apr 08, 2003 9:51 pm
- Location: Australia
Hi Kduke,
Thanks for your info.One more thing to add.If I increase the NOFILES in Sunsolaris 2.8 will there be any impact in OS(like kernel parameter to be tunned or changed).
Following is my Config
$ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 8192
coredump(blocks) unlimited
nofiles(descriptors) 256
memory(kbytes) unlimited
Thanks
-Chandru
Thanks for your info.One more thing to add.If I increase the NOFILES in Sunsolaris 2.8 will there be any impact in OS(like kernel parameter to be tunned or changed).
Following is my Config
$ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 8192
coredump(blocks) unlimited
nofiles(descriptors) 256
memory(kbytes) unlimited
Thanks
-Chandru
What do you mean? Sun Solaris 2.8 is the OS. It is also UNIX. NOFILES is also a UNIX kernel parameter. You need to change MFILES to 200 or more which should be plenty. NOFILES is large enough. Do you have jobs with more than 100 links? There is another parameter for the number of DYNAMIC hash files. If you edit uvconfig then you will see it in the comments. I would up this as well. Do a search Ray has covered changing these several times. He has a step by step post to walk you through changing these.
Mamu Kim
-
- Charter Member
- Posts: 13
- Joined: Tue Apr 08, 2003 9:51 pm
- Location: Australia