Very Slow performance - Table to table data transfer

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
SonShe
Premium Member
Premium Member
Posts: 65
Joined: Mon Aug 09, 2004 1:48 pm

Very Slow performance - Table to table data transfer

Post by SonShe »

I have a job that moves data from one oracle table to another oracle tables. The datastage server and oracle server are physically different. I am getting a speed of only 100 rows per second even if it is a simple transfer of data.

I have OCI -> TRANSFORMER -> OCI stages. The transaction size has been set to zero. The array size is 1000. There are about 7 million rows to be transferred.

This kind of volume processing is only a one-time event. I will appreciate any ideas or help. I would like to know why the simple transfer speed is so low. I get very good speed in other jobs that involve much more data transformation.

Thanks.
poorna_76
Charter Member
Charter Member
Posts: 190
Joined: Thu Jul 08, 2004 10:42 am

Re: Very Slow performance - Table to table data transfer

Post by poorna_76 »

SonShe wrote:I have a job that moves data from one oracle table to another oracle tables. The datastage server and oracle server are physically different. I am getting a speed of only 100 rows per second even if it is a simple transfer of data.

I have OCI -> TRANSFORMER -> OCI stages. The transaction size has been set to zero. The array size is 1000. There are about 7 million rows to be transferred.

This kind of volume processing is only a one-time event. I will appreciate any ideas or help. I would like to know why the simple transfer speed is so low. I get very good speed in other jobs that involve much more data transformation.

Thanks.
Try to use any Bulk Loaders Stages.

Also try to incorporate IPC stage/stages in your job design, to increase the performance.
You can even specify this at project level.

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Are there any 'transformations' involved? If not, I wouldn't even use DataStage for something like this.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply