fatal error - sort stage

vij · Post by **vij** » Wed Dec 20, 2006 11:39 pm

this is the error i am getting after i ran the job:

APT_CombinedOperatorController(0),0: Failure during execution of operator logic.
APT_CombinedOperatorController(0),0: Fatal Error: Tsort merger aborting: Scratch space full

is this related with the memory of sort stage? actually i am sorting based on two fields and the input number of rows is 8055979.

pls help in solving the error.

i have not used any partitioning methods on this stage ...shuld i use any?

ray.wurlod · Post by **ray.wurlod** » Thu Dec 21, 2006 12:50 am

You can configure the maximum memory (per partition) that the Sort stage will use (by default it is 30MB). Any more than this spills to the scratch disk nominated in the configuration file.

Partitioning is irrelevant, except that a poorly chosen partitioning algorithm (for example one that forces all rows onto one partition) will exacerbate the problem.

What you must do is some combination of allowing the Sort stage to use more memory and allocating additional scratchdisk resource.

vij · Post by **vij** » Thu Dec 21, 2006 1:01 am

ray.wurlod wrote:You can configure the maximum memory (per partition) that the Sort stage will use (by default it is 30MB). Any more than this spills to the scratch disk nominated in the configuration file.

Partit ...

how to avoid this error? is it possible to change the maximum capacity?how?if i use partitioning (i have used HASH partition now) will the same be avoided?

vij · Post by **vij** » Thu Dec 21, 2006 1:18 am

vij wrote:
ray.wurlod wrote:You can configure the maximum memory (per partition) that the Sort stage will use (by default it is 30MB). Any more than this spills to the scratch disk nominated in the configuration file.

Partit ...
how to avoid this error? is it possible to change the maximum capacity?how?if i use partitioning (i have used HASH partition now) will the same be avoided?

now, i get these fatal errors from sorter - APT_CombinedOperatorController,2: write failed: Output file full, and no more output files
APT_CombinedOperatorController,2: Failure during execution of operator logic.
APT_CombinedOperatorController,2: Fatal Error: Tsort merger aborting: mergeOneRecord() punted
cs}}},2: Failure during execution of operator logic
APT_CombinedOperatorController,2: Fatal Error: Pipe read failed: short read.

thompsonp · Post by **thompsonp** » Thu Dec 21, 2006 2:35 am

Look closely at the error messages.

Your first post highlighted that the Scrtach space was full.
As Ray said the location of the Scratch space is defined in your configuration file and is used to write data that is being sorted and doesn't fit in memory.

As suggested you could adjust the optional parameter 'Restrict Memory Usage' on the sort stage from the default value of 20MB to a greater value - provided you have enough memory, so that there is less space required in the Scratch area.

Your last post indicates that your Output File is full. This message has come from a combined controller. Add the environment variable APT_DISABLE_COMBINATION and set it to TRUE and rerun the job. The message you get should indicate which stage has run out of space. Be sure to remove this environment variable when you have things working.

I would guess that your are writing a file or dataset somewhere and it is probably to the same disk as your scratch space. Check the available space at these locations.

ray.wurlod · Post by **ray.wurlod** » Thu Dec 21, 2006 3:07 am

Both sets of error messages indicate that you're running out of disk space. If you're trying to write to a Data Set, File Set or Lookup File Set that's the disk resource defined in the configuration file; if not, then it's the file system on which your file exists.

The solution: add more disk space. More file systems are needed; not more directories on the same file system.

keshav0307 · Post by **keshav0307** » Thu Dec 21, 2006 6:51 am

the sort stage create some temporary file it tmpdir; so make it sure it has enough sapce.. almost double (in total of all node servers) the size of data you are sorting.