DSXchange

Hi,

I went thru couple of posts on the same problem as this http://dsxchange.com/viewtopic.php?t=93 ... a4bea76809.

Problem: Not able to create a sequential file more than 1.5 gb and aborts with the message "File too large".

The "ulimit -a;id" from DS before job routine results as

Time(seconds) unlimited
file(blocks) 2097151
data(kbytes) 131072
stack(kbytes) 32768
memory(kbytes) 32768
coredump(blocks) 2097151
nofiles(descriptors) 2000
uid=208(dsadm) gid=208(dstage) groups=1(staff)

At AIX level for dsadm,

time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 245760
stack(kbytes) 16384
memory(kbytes) unlimited
coredump(blocks) unlimited
nofiles(descriptors) 2000

Even after bouncing the DS server after the ulimit was changed at the OS level, the DS log message shows the old value.

Environment : DS 7.5.2 on AIX 5.3

I could create a file > 2gb at os level.

Please help.

Thanks!

I think that your ulimit settings for root are limited, since that user spawns the other processes this is where you limitations are inherited from at runtime.

Hi Arnd,

Since i confirmed that the user dsadm has unlimited privilege for fileblocks at OS level, how does the root's privilege interfere with DS execution for creating large files using the user dsadm? Could you please elaborate.

Thanks

please re-read my post. I think it is the "root" user causing the issue. If a process has ulimits and spawns another, then the new one cannot have a higher ulimit (even if the settings are unlimited) than the parent.

Hi Arndw,

If the root user ulimit settings are overriding the dsadm user's setting, then how are files > 2gb could be created using dsadm at the back end?

thanks

Hi ArndW,

Just to add, even after changing the root setting of fileblocks to unlimited, the problem persists.

Thanks

He is saying the override only happens during job execution, not all the time. What are the ulimit settings for root on that box?

Ok, I see you've changed them. Did you stop and restart DataStage after making the change?

Hi,

I have changed the fileblocks for root user to unlimited and restarted the DS server. I still find the ulimit for dsadm in DS log shows 2 gb though it is unlimited at OS level.

Please help.

Thanks!

Strange. Did you restart DataStage as root or as your dsadm user? Also, check your $DSHOME/dsenv file to see if it contains a ulimit command.

Hi ArndW,

I know its really weird, but i dont know how to resolve this.

I restarted the DS server as dsadm. There are no entries for ulimit in dsenv file though there are entries in uvconfig file which you know will not impact if the root and dsadm has unlimited settings for fileblocks.

Thanks

If you have root access try to run your job and see what the ulimit settings display.

Also, if you execute ". $DSHOME/dsenv" as dsadm do the ulimit settings remain the same before & after the command?

Known Facts
========
1. root user has unlimited ulimit
2. dsadm user has unlimited ulimit
3. $DSHOME/dsenv contains no ulimit command
4. System and filesystem limits allow creation of files >2Gb by dsadm/root from OS
5. Display of "ulimit -a" in before job shows limited settings for dsadm user

Hi,

I do not have the root access, but i confirmed with my admin that he has granted unlimited to all properties of ulimit.

I executed ulimit -a for dsadm at os level before and after executing dsenv. the results are the same. I confirm the known facts which you have mentioned.

thanks!

Hi,

I do not have the root access, but i confirmed with my admin that he has granted unlimited to all properties of ulimit.

I executed ulimit -a for dsadm at os level before and after executing dsenv. the results are the same. I confirm the known facts which you have mentioned.

thanks!

Would you be able to get someone with root login to try logging into DataStage at your workstation and test the runtime limit? Also, could you try the following from UNIX Is the ulimit still unlimited inside the DS command environment?

At some point soon here, I would think you'd be better served by opening a case with your official Support provider rather than continuing to thrash here.