Job aborting due to memory issue

parag.s.27 · Post by **parag.s.27** » Thu Dec 20, 2007 12:11 am

Hi All,

After reading many posts in the forum and trying many options given, My job is still failing due to insufficient memory.

The error I am getting is: -

Code: Select all

Lkp_CoverDetail_Ins_Upd,0: Could not map table file "/u03/Datasets/lookuptable.20071219.suos0lb (size 1700927896 bytes)": Not enough space

Lkp_CoverDetail_Ins_Upd,0: Error finalizing / saving table /u04/tmp/dynLUT95869231003dcb

Lkp_CoverDetail_Ins_Upd,0: The runLocally() of the operator failed.

The server configuration is: -
RAM - 16 GB
Disk space -
/u03/Datasets/ - 260 GB
/u04 - 40 GB
4 Processors

Over all total disk space - 300 GB

I also tried the command ulimit -s {Size}.

The setting is

Code: Select all

ulimit -s unlimited

. We tried on 2 different servers but the job aborts at the same point. The IBM people are saying that its not a space issue. Can anyone please help in this regard

parag.s.27 · Post by **parag.s.27** » Thu Dec 20, 2007 12:15 am

Sorry to mention in earlier post: -

There are 40 million plus records comming in. Also I had to use lookup because business wants to use the data of reject link. and hence I had to use lookup stage so that, the reject records will be used by business

balajisr · Post by **balajisr** » Thu Dec 20, 2007 12:29 am

Use Join Stage to do left outer join and reject the rows which do not satisfy the join criteria in the downstream transformer or filter stage.

ray.wurlod · Post by **ray.wurlod** » Thu Dec 20, 2007 1:42 am

The "not enough space" is in /u04/tmp.

parag.s.27 · Post by **parag.s.27** » Fri Dec 21, 2007 4:50 am

ray.wurlod wrote:The "not enough space" is in /u04/tmp. ...

Hi,

we were monitoring the memory and disk usage while running the job.

What we found out is, The temp file getting created is of 4 GB size, also it created 300 partitions in the disk, but we could not monitor the RAM usage.

Also the problem is if I use a joiner, then to get the reject link data I'll have to use an extra transformer stage and will have to do a left outer join on entire database.

ArndW · Post by **ArndW** » Fri Dec 21, 2007 5:09 am

This looks suspiciously like the problem I was asked to look at yesterday. Was the 2-node APT_CONFIG file that I recommended used, one which cross-mapped the /u03 and /u04 partitions for dataset and scratch directories?
What does "300 partitions on the disk" mean?
Also, why do you say that the job is failing because of insufficient memory? That is not the case and certainly not shown in the log files.

ramesh_inform · Post by **ramesh_inform** » Mon Jan 28, 2008 5:20 am

how about increasing the size of APT_BUFFER_MEMORY and APT_MAX_MEMORY?

ramesh_inform · Post by **ramesh_inform** » Mon Jan 28, 2008 5:21 am

how about increasing the size of APT_BUFFER_MEMORY and APT_MAX_MEMORY?

ray.wurlod · Post by **ray.wurlod** » Mon Jan 28, 2008 9:52 pm

... but not unintelligently!