Lookups and Performance with ODBC stages
Posted: Thu Mar 02, 2006 2:45 am
Hi guys,
we're implementing several extracting jobs against DB2 databases. During the process there are several joins to be implemented. In the first attempt of creating a single lookup, the primary link was a select with a ODBC the reference link was a previously created Hash File (with the preload to memory option set to true) and the output was an insert to a ODBC stage. I read 200 rows/sec from the initial ODBC stage but the lookup was slowing the perfomance to 8 rows/sec. I think this is very slow, the average performance was 8 rows/sec, Is that normal??
In terms of performance whatg is better to do lookups or to implement the JOINS in the SQL user-defined of the ODBC stage?? Doi you guys have any guidelines for that??
Thanks in advance.
Julio
we're implementing several extracting jobs against DB2 databases. During the process there are several joins to be implemented. In the first attempt of creating a single lookup, the primary link was a select with a ODBC the reference link was a previously created Hash File (with the preload to memory option set to true) and the output was an insert to a ODBC stage. I read 200 rows/sec from the initial ODBC stage but the lookup was slowing the perfomance to 8 rows/sec. I think this is very slow, the average performance was 8 rows/sec, Is that normal??
In terms of performance whatg is better to do lookups or to implement the JOINS in the SQL user-defined of the ODBC stage?? Doi you guys have any guidelines for that??
Thanks in advance.
Julio