Ulimit problem
Moderators: chulett, rschirm, roy
Ulimit problem
Hi,
I went thru couple of posts on the same problem as this http://dsxchange.com/viewtopic.php?t=93 ... a4bea76809.
Problem: Not able to create a sequential file more than 1.5 gb and aborts with the message "File too large".
The "ulimit -a;id" from DS before job routine results as
Time(seconds) unlimited
file(blocks) 2097151
data(kbytes) 131072
stack(kbytes) 32768
memory(kbytes) 32768
coredump(blocks) 2097151
nofiles(descriptors) 2000
uid=208(dsadm) gid=208(dstage) groups=1(staff)
At AIX level for dsadm,
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 245760
stack(kbytes) 16384
memory(kbytes) unlimited
coredump(blocks) unlimited
nofiles(descriptors) 2000
Even after bouncing the DS server after the ulimit was changed at the OS level, the DS log message shows the old value.
Environment : DS 7.5.2 on AIX 5.3
I could create a file > 2gb at os level.
Please help.
Thanks!
I went thru couple of posts on the same problem as this http://dsxchange.com/viewtopic.php?t=93 ... a4bea76809.
Problem: Not able to create a sequential file more than 1.5 gb and aborts with the message "File too large".
The "ulimit -a;id" from DS before job routine results as
Time(seconds) unlimited
file(blocks) 2097151
data(kbytes) 131072
stack(kbytes) 32768
memory(kbytes) 32768
coredump(blocks) 2097151
nofiles(descriptors) 2000
uid=208(dsadm) gid=208(dstage) groups=1(staff)
At AIX level for dsadm,
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 245760
stack(kbytes) 16384
memory(kbytes) unlimited
coredump(blocks) unlimited
nofiles(descriptors) 2000
Even after bouncing the DS server after the ulimit was changed at the OS level, the DS log message shows the old value.
Environment : DS 7.5.2 on AIX 5.3
I could create a file > 2gb at os level.
Please help.
Thanks!
I think that your ulimit settings for root are limited, since that user spawns the other processes this is where you limitations are inherited from at runtime.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
please re-read my post. I think it is the "root" user causing the issue. If a process has ulimits and spawns another, then the new one cannot have a higher ulimit (even if the settings are unlimited) than the parent.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Strange. Did you restart DataStage as root or as your dsadm user? Also, check your $DSHOME/dsenv file to see if it contains a ulimit command.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
If you have root access try to run your job and see what the ulimit settings display.
Also, if you execute ". $DSHOME/dsenv" as dsadm do the ulimit settings remain the same before & after the command?
Known Facts
========
1. root user has unlimited ulimit
2. dsadm user has unlimited ulimit
3. $DSHOME/dsenv contains no ulimit command
4. System and filesystem limits allow creation of files >2Gb by dsadm/root from OS
5. Display of "ulimit -a" in before job shows limited settings for dsadm user
Also, if you execute ". $DSHOME/dsenv" as dsadm do the ulimit settings remain the same before & after the command?
Known Facts
========
1. root user has unlimited ulimit
2. dsadm user has unlimited ulimit
3. $DSHOME/dsenv contains no ulimit command
4. System and filesystem limits allow creation of files >2Gb by dsadm/root from OS
5. Display of "ulimit -a" in before job shows limited settings for dsadm user
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Would you be able to get someone with root login to try logging into DataStage at your workstation and test the runtime limit? Also, could you try the following from UNIX
Is the ulimit still unlimited inside the DS command environment?
Code: Select all
cd $DSHOME
. ./dsenv
bin/dssh {This will put you into the DS shell}
SH -c "ulimit -a"
QUIT {This will bring you back out to your shell}
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>