Page 1 of 2

ODBC Stage Speed

Posted: Fri May 29, 2009 9:02 am
by sgubba
I am tryin to load data using ODBC stage. I am wusing Upsert mode and what i realise is its extremely slow some times close to 37 rows per second is there any way i can improve the speed

Thanks

Shyam

Posted: Fri May 29, 2009 9:09 am
by chulett
Is 'upsert' really necessary? Can you use a native Enterprise stage instead? Would help to mention the target db as well.

Posted: Fri May 29, 2009 9:20 am
by sgubba
Yep, I need to capture rejects so i am using upsert. The target is ORACLE 11g Since we dont have drivers we are using ODBC

Shyam

Posted: Fri May 29, 2009 9:25 am
by chulett
Are you doing the upserts in the 'right' order, meaning whichever action has the best chance of succeeding first? For any updates, are you updating on indexed columns?

Posted: Fri May 29, 2009 10:07 am
by sgubba
Actually all are inserts because i need to capture rejects. I am using upsert, As reject option is not available in Write

Shyam

Posted: Fri May 29, 2009 10:11 am
by chulett
Okay... Array Size?

Posted: Fri May 29, 2009 10:22 am
by sgubba
60000

Posted: Fri May 29, 2009 10:46 am
by chulett
Seems... high. What is your ARL, average record length? Does it help if you drop it down to a smaller number, say like 500 or 1000?

Posted: Fri May 29, 2009 12:36 pm
by sgubba
Its barely 7-8 columns which amounts to 100 -150 bytes

Posted: Fri May 29, 2009 1:04 pm
by chulett
Still... did you try other values?

Posted: Fri Jun 19, 2009 8:15 am
by sgubba
Yeah,

The record length is really small. Its extremely slow ...I have no idea what to do..please suggest to improve the performance

Shyam

Posted: Fri Jun 19, 2009 8:21 am
by miwinter
Is obtaining the Oracle drivers so much of a hurdle? Strikes me you'd be better off doing what Craig originally suggested and use a native Enterprise stage as opposed to ODBC, if performance is your 'driver' (sorry!) :D

Though you say "all are inserts", is that just by nature of your processing, or have you actually set "insert only" or "insert then update" on your upsert mode?

Posted: Fri Jun 19, 2009 9:12 am
by sgubba
Insert Only

Posted: Fri Jun 19, 2009 9:14 am
by sgubba
I am trying to insert in to Oracle 11g. Thts the reason we are using ODBC

Posted: Fri Jun 19, 2009 9:20 am
by miwinter
So there are no drivers for Oracle 11g then?