Page 1 of 1

problem with Aggregator stage.

Posted: Sun Oct 18, 2009 5:15 am
by zulfi123786
"Aggregator_91,0: The current soft limit on the data segment (heap) size (2147483645) is less than the hard limit (2147483647), consider increasing the heap size limit"
This is the error i am getting.... Please advice how to overcome this.

Posted: Sun Oct 18, 2009 6:22 am
by ArndW
Is this a warning message or an error message causing the job to abort? One option you have is to sort your incoming data according to your aggregation group(s).

Posted: Sun Oct 18, 2009 12:21 pm
by zulfi123786
this is an error message which is causing the job to abort

Posted: Mon Oct 19, 2009 2:55 am
by ArndW
In that case your memory requirements for this stage are higher than what the machine can give you. Since you cannot increase your hard data segment size you will need to reduce your storage requirements, which can be done in this case by sorting your data before the aggregator stage.