Hi,
I am extracting data from Oracle 8i using OraOCI8.Then after making some transformations,I do a lookup for dimension keys in the corresponding HashFiles.After that I am Aggregating the Data using an Aggregator Stage.
The total number of rows extracted from source is around 9000000.While aggregating the Aggregator Stage terminates abruptly after processing around 2700000 records giving a message " Abnormal Termination Of Aggregator Stage.It does not give any other error message.
Please suggest.
Aggregator Stage Tremination
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You may get an error event logged if you reset the job (rather than recompiling it). Otherwise you can search in the &PH& directory on the server.
The usual cause for aborting an Aggregator stage is that it needs more memory than is available for it to allocate.
Try sorting your data by the column(s) that will be grouped (include an ORDER BY clause in the Oracle SELECT statement), and changing the input properties of the Aggregator stage to reflect the fact that it is sorted by the grouping columns.
This will allow the Aggregator stage to operate much more efficiently (maybe 10-20 times faster), and not abort when it runs out of memory.
The usual cause for aborting an Aggregator stage is that it needs more memory than is available for it to allocate.
Try sorting your data by the column(s) that will be grouped (include an ORDER BY clause in the Oracle SELECT statement), and changing the input properties of the Aggregator stage to reflect the fact that it is sorted by the grouping columns.
This will allow the Aggregator stage to operate much more efficiently (maybe 10-20 times faster), and not abort when it runs out of memory.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.