Aggregator Stage Tremination

bapajju · Post by **bapajju** » Wed Dec 10, 2003 3:46 am

Hi,
I am extracting data from Oracle 8i using OraOCI8.Then after making some transformations,I do a lookup for dimension keys in the corresponding HashFiles.After that I am Aggregating the Data using an Aggregator Stage.
The total number of rows extracted from source is around 9000000.While aggregating the Aggregator Stage terminates abruptly after processing around 2700000 records giving a message " Abnormal Termination Of Aggregator Stage.It does not give any other error message.
Please suggest.

ray.wurlod · Post by **ray.wurlod** » Wed Dec 10, 2003 4:29 am

You may get an error event logged if you reset the job (rather than recompiling it). Otherwise you can search in the &PH& directory on the server.
The usual cause for aborting an Aggregator stage is that it needs more memory than is available for it to allocate.
Try sorting your data by the column(s) that will be grouped (include an ORDER BY clause in the Oracle SELECT statement), and changing the input properties of the Aggregator stage to reflect the fact that it is sorted by the grouping columns.
This will allow the Aggregator stage to operate much more efficiently (maybe 10-20 times faster), and not abort when it runs out of memory.

peterbaun · Post by **peterbaun** » Wed Dec 10, 2003 4:32 am

Hi -

Always difficult to find an error based on this kind of error message.
My best suggestion is to check for null-values

Regards
Peter

bapajju · Post by **bapajju** » Wed Dec 10, 2003 6:15 am

Thanks a ton Ray.It worked afer sorting.It was a memorry problem only.Appreciate your help.