ulimit -a shows different with the same id

shin0066 · Post by **shin0066** » Tue Dec 18, 2007 2:10 pm

Hi gurus,
we are having Heap size allocation errors - we have 2 TB of scratch space but still jobs are failing when we run huge volume of data.

I read in message board to look for the ulimit for the id which i am using to run the jobs.
on unix command line ulimit -a shows

Code: Select all

time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        unlimited
memory(kbytes)       unlimited
coredump(blocks)     unlimited
nofiles(descriptors) 2000

but where as when i run a routine in Datastage as
Call DSExecute ("UNIX", "ulimit -a", Output, SystemReturnCode)

it shows different stats as below

Code: Select all

 time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         786432
stack(kbytes)        unlimited
memory(kbytes)       unlimited
coredump(blocks)     unlimited
nofiles(descriptors) 2000

data(kbytes) are different for the same id - why is like that..?

Any info is appreciated.

ray.wurlod · Post by **ray.wurlod** » Tue Dec 18, 2007 4:03 pm

It's not the same user. Look at the environment variables when running a job, to determine the user ID actually running the job. Adjust that user's ulimit.

It may also be the case that the ULIMIT configuration parameter in uvconfig comes into play, something else that may be worth checking.

Finally do you have a ulimit command in the dsenv script? It is unusual, but might serve to reduce process ulimit (a process other than superuser can not increase a ulimit).

shin0066 · Post by **shin0066** » Tue Dec 18, 2007 6:12 pm

Hi Ray,

i used the same user id to run the command at UNIX command level and logged to DataStage with same ID and executed the same via a routine.

And i did check ULIMIT in uvconfig file it is set to 128000, but i am not sure it is overriding the value while running the ulimit -a via datastage routine.

also checked in dsenv file - there is no ulimit set in it.

Thanks,

ray.wurlod · Post by **ray.wurlod** » Tue Dec 18, 2007 7:40 pm

ULIMIT in uvconfig only comes into play if the UNIX ulimit is lower. Here it isn't. Did you check the job log entry that contains the environment variables, to verify under what user ID the job actually executes?

shin0066 · Post by **shin0066** » Wed Dec 19, 2007 10:53 am

Hi Ray,

here is what i see from the job log for environment variables

Code: Select all

Environment variable settings:
_=/usr/bin/nohup
LANG=C
LOGIN=dsadm
LDR_CNTRL=MAXDATA=0x30000000
LOGNAME=user123
.
.
.
.
.APT_CONFIG_FILE=/dsadm/Ascential/DataStage/Configurations/Proj.apt
APT_DEFAULT_TRANSPORT_BLOCK_SIZE=32768
APT_MONITOR_MINTIME=10
APT_STRING_PADCHAR= 
.
.
.
.
.

I verified dsadm id's ulimit via unix command line as below - still different

Code: Select all

time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        unlimited
memory(kbytes)       unlimited
coredump(blocks)     unlimited
nofiles(descriptors) 2000

Same id via datastage routine

Code: Select all

time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         786432
stack(kbytes)        unlimited
memory(kbytes)       unlimited
coredump(blocks)     unlimited
nofiles(descriptors) 2000

Any help is appreciated

Thanks

ray.wurlod · Post by **ray.wurlod** » Wed Dec 19, 2007 4:14 pm

Routines run from the test grid are executed by dsapi_slave. It is that user whose data stack size is limited. You can not infer anything from this. You need to execute the ulimit -a command from within a running job (perhaps as a before-job subroutine via ExecSH) to find out what's really happening.

shin0066 · Post by **shin0066** » Wed Dec 19, 2007 5:35 pm

Hi Ray,

I ran the ulimit -a from Before job routine and i see the below information which produces same as from routine in my previous case.

Code: Select all

time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         786432
stack(kbytes)        unlimited
memory(kbytes)       unlimited
coredump(blocks)     unlimited
nofiles(descriptors) 2000

I talked to my Unix Admin to bump up everything to Unlimited except nofiles and he did - when i do ulimit -a from command line it is showing all unlimited except nofiles part but if i run ulimit -a via a shell script as above. Why the difference for command line and shell script?

Thanks

ray.wurlod · Post by **ray.wurlod** » Wed Dec 19, 2007 6:12 pm

OK, try running ulimit -a ; id from the before-job subroutine. The id command may yield a clue.

shin0066 · Post by **shin0066** » Sat Dec 22, 2007 8:12 am

Hi Ray,

my unix admin set all ulimit parameters to unlimited as hard limits, but somehow the data segment value is overwritten via the dsenv profile and i am sure there is not set up for ulimit in our dsenv file. The reason i am telling because it is coming from dsenv profile is - i tested the scenario by removing the dsenv profile sourcing from my id .profile and it showed unlimited for all the ulimit parameters.

I read few message posted by you that we can set the ulimit in dsenv profile to unlimited.

Can you please provide the example and command to set data segment ulimit to unlimited?

Thanks

shin0066 · Post by **shin0066** » Sat Dec 22, 2007 8:20 am

Hi Ray,

my unix admin set all ulimit parameters to unlimited as hard limits, but somehow the data segment value is overwritten via the dsenv profile. I am sure there is nothing set for ulimit in our dsenv file.

The reason I am telling because it is coming from dsenv profile is - I tested the scenario by removing the dsenv profile sourcing from my id .profile and it showed unlimited for all the ulimit parameters.

I read few message posted by you that we can set the ulimit in dsenv profile to unlimited.

Can you please provide the example and command to set data segment ulimit to unlimited?

Thanks

ray.wurlod · Post by **ray.wurlod** » Sat Dec 22, 2007 3:45 pm

shin0066 wrote:I read few message posted by you that we can set the ulimit in dsenv profile to unlimited.

Can you please provide the example and command to set data segment ulimit to unlimited?

Show me where. I doubt that I ever did.

Only superuser can increase ulimit.

What is the ulimit for the dsrpcd process?

shin0066 · Post by **shin0066** » Sun Dec 23, 2007 10:10 am

Hi Ray,

Here is the posting viewtopic.php?t=115283&highlight=data+segment

currently job is aborting as following error

Code: Select all

 The current soft limit on the data segment (heap) size (805306368) is less than the hard limit (2147483647), consider increasing the heap size limit

Also noticed in dsenv file - we are setting memory as
LDR_CNTRL=MAXDATA=0x30000000;export LDR_CNTRL

but 805306368 is equal to 768 MB which should be 2 times of data segments for a process - we are setting it to 3 times in dsenv file it should be 2 GB.

By chance, does this causing to set the data segment not to unlimited?

Thanks for you time

ray.wurlod · Post by **ray.wurlod** » Sun Dec 23, 2007 2:18 pm

Ah. That wasn't me, that was ArndW.

You still haven't reported the ulimit for dsrpcd nor the value of USER or LOGNAME environment variable from a job run.

ArndW · Post by **ArndW** » Mon Dec 24, 2007 4:57 am

Have you done both "ulimit -Sa" and "ulimit -Ha"? Also, have you tried "ulimit -Ss 2147483647"?

shin0066 · Post by **shin0066** » Fri Dec 28, 2007 11:22 am

Hi ArndW,

I haven't test with with option you suggested. Do i need to set that in my profile or dsenv profile?

Ray, can you please tell me how to see ulimit for dsrpc? i checked dsrpc using ps -ef | grep dsrpc and it is running as root.

Thanks