Fork Error - DataStage resource Limitations

[shin0066]

Hi Ray,
I found the reply to one of Fork error issues - You mentioned to add a call to DSExecute() to invoke a ulimit - how do i do that? Any information is appreciated.

No but you can. Add a call to DSExecute() to invoke ulimit -a and write the result into an informational log entry. This will let you see all the resource limits to which the DataStage process is subject.

Thanks

[ray.wurlod]

Code: Select all

Shell = "UNIX"
Command = "ulimit -a"
Output = ""
ExitStatus = -1
Call DSExecute(Shell, Command, Output, ExitStatus)
Call DSLogInfo("ulimit -a" : @FM : Output, "Limits")

[shin0066]

Thanks Ray,

When I did Ulimit -a at command line with production id which we are running etl job here is the result

time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) unlimited
memory(kbytes) unlimited
coredump(blocks) unlimited
nofiles(descriptors) 2000

Where are i ran the same Ulimit command via routine, here is the output.
> time(seconds) unlimited
> file(blocks) unlimited
> data(kbytes) 786432
> stack(kbytes) unlimited
> memory(kbytes) unlimited
> coredump(blocks) unlimited
> nofiles(descriptors) 2000

how does it make a difference to create Fork Errors while i am ETL jobs?

Thanks

[ray.wurlod]

It doesn't. I answered the question you asked, which was how to invoke ulimit from DSExecute.

fork() is usually hampered by the UNIX process table becoming full, or an attempt being made to start (fork) too many child processes simultaneously system wide, which causes a bottleneck on the process table.

There are other possible causes, but that one is the most likely.