Memory consumption by DataStage for processing

[vnspn]

Hi,

We would like to know, when DataStage Server is being installed, would we need to specify the "amount of memory value" that DataStage Server can use for processing a Job.

Lets say if the server box has 8 GB of memory, would DataStage make use of all avaialble memory while processing a Job?

For instance, we transform 10 million records of data from a Flat file source to an Oracle table in a Job. At the time of execution, DataStage process a bunch of records at a time (i.e., 'X' no. of records per sec it shows on the performance statistics). What determines the DataStage to decide to process 'X' no. of records at a time? Is it based on the memory available on the Server?

Also at any point in time of processing, where does DataStage hold all these records? Is it on the Server's RAM?

Thanks.

[ray.wurlod]

DataStage will request what memory it needs. In general data are not held in memory, but streamed through the job. Exceptions include blocking stages, such as the Aggregator and Sort stages. If the requested (virtual) memory can not all be allocated, then you are likely to receive a "heap allocation failed" error.

Records/second is a totally meaningless metric in the context of this question. Ignore it.

The number of records at a time is influenced only by the buffer sizes that you set if you enable row buffering.

Note that this answer applies only to server jobs; a completely different answer applies to parallel jobs.