Memory Error
Posted: Thu Jun 04, 2009 3:30 pm
Hi,
The job process around 10,000,000 records. There are 5 columns and there data types are very small
Update Action: Insert new rows or update existing rows
I have tried Array Size 1 and Transaction size 1 to 1000, still no luck
When the job was running, there was enough space in Scratch disk.
Design:
I have used APT_DISABLE_COMBINATION and set it values to True, the error is shown in the DRS_STAGE.
Error:
When I monitored the memory usage from the top command:
Is the Virtual Memory should grow until the job finishes? As this is using most of the memory, I don't want increase the APT_BUFFER* size and crash the system..
Is there any way the size can be optimized?
Thanks
Prasad
The job process around 10,000,000 records. There are 5 columns and there data types are very small
Update Action: Insert new rows or update existing rows
I have tried Array Size 1 and Transaction size 1 to 1000, still no luck
When the job was running, there was enough space in Scratch disk.
Design:
Code: Select all
Oracle --> Transformer --> DRS[MS SQL Server]
Error:
Code: Select all
DRS_STAGE,0: clntudp_create: out of memory
DRS_STAGE,0: clntudp_create: out of memory
DRS_STAGE,0: clntudp_create: out of memory
DRS_STAGE,0: Operator terminated abnormally: received signal SIGSEGV
Code: Select all
VIRTUAL MEMORY was 4057M and RES is 3.9G when the job aborted..
Where VIRT is Virtual Size of the task. This includes the size of process's executable binary, the data area and all the loaded shared libraries.
Where RES is Resident memory: current amount of process memory that resides in physical memory
Is there any way the size can be optimized?
Thanks
Prasad