OPTIMIZATION

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
karry450
Participant
Posts: 201
Joined: Wed Nov 14, 2007 11:31 pm
Location: HYD

OPTIMIZATION

Post by karry450 »

Hi Friends,

I am running a server job in which my source is flatfile and target is oracle.

I have tried normal loading and bulk loading in which normal loading is faster than bulk loading.


Is there anyother way where i can increase the speed of loading records and performance also.

Can anyone suggest me please.

Thanks and regards
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Define "performance" in an ETL context.

Try writing to a text file, and using a customized sqlldr control file to load into Oracle from that. For example, get the buffer sizes right (rather than defaulting to 256 for every field), use parallel load capabilities, and so on.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
karry450
Participant
Posts: 201
Joined: Wed Nov 14, 2007 11:31 pm
Location: HYD

Optimization

Post by karry450 »

ray.wurlod wrote:Define "performance" in an ETL context.

Try writing to a text file, and using a customized sqlldr control file to load into Oracle from that. For example, get the buffer sizes right (rather than d ...

Thanks , I have tried selecting Inprocess and defining some buffer 128 and tried this way, it is bit faster than normal loading.

But is there? any other way which gives lot of difference.


Thanks.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Yes, the way I suggested.

Premium membership is one of the ways that the hosting and bandwidth costs incurred by DSXchange are met. At less than 30c (Rs12) per day, it is definitely a worthwhile investment for anyone claiming to be a DataStage practitioner.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

DIRECT path or CONVENTIONAL? Lots of indexes or none? Triggers? Partitioned table? There are so many factors. Please narrow your question and describe the table and method of load.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Not sure why people seem loath to do this, but go have a chat with your DBA. They LOVE helping with stuff like this. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply