Posted: Mon Dec 30, 2002 11:55 pm
The Aggregator stage is necessarily memory-hungry. The amount of memory used for its in-memory tables can be configured from the DS.TOOLS menu.
Far better, though, is to minimize the amount of memory demanded by the stage, by sorting its input on the grouping columns (and, of course, not passing any columns through the stage that you don't need to!). If the grouping columns are sorted, and asserted (in the input link properties) to be sorted, then memory can be freed each time one of the sorted grouping columns changes value. This also makes the Aggregator stage not block until all rows are processed.
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
Far better, though, is to minimize the amount of memory demanded by the stage, by sorting its input on the grouping columns (and, of course, not passing any columns through the stage that you don't need to!). If the grouping columns are sorted, and asserted (in the input link properties) to be sorted, then memory can be freed each time one of the sorted grouping columns changes value. This also makes the Aggregator stage not block until all rows are processed.
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518