We need to use the correct optimized method in our ODBC stage.
Our environment-
1. We read from two types of tables - source, ref tables.
Our source table gets loaded with 2k rows everyday and cleaned every 2 months(120k max accumulation in 2months). Our REF tables wouldn't exceed 1k rows.
2. There are two ways to extract from the source table. Which is the best method?
a. Use a table READ method in the ODBC stage and use a filter stage.
b. Write a query with a WHERE clause and use User defined SQL.
Our WHERE clause consists of only one column with a status code =0/1 check.
3. During the lookups again Read Method = Table vs User def SQL.
a. Does a Read method = table cache the table into memory and avoid a lookup against the table row by row?
b. Everytime we lookup against the REF table we do it by row by row? Correct?
All responses are greatly appreciated. Thanks!
ODBC Read method=Table vs User def SQL
Moderators: chulett, rschirm, roy
We need to use the correct optimized method in our ODBC stage.
Our environment- 4 CPUs Solaris.
1. We read from two types of tables - source, ref tables.
Our source table gets loaded with 2k rows everyday and cleaned every 2 months(120k max accumulation in 2months). Our REF tables wouldn't exceed 1k rows.
2. There are two ways to extract from the source table. Which is the best method?
a. Use a table READ method in the ODBC stage and use a filter stage.
b. Write a query with a WHERE clause and use User defined SQL.
Our WHERE clause consists of only one column with a status code =0/1 check.
3. During the lookups again Read Method = Table vs User def SQL.
a. Does a Read method = table cache the table into memory and avoid a lookup against the table row by row?
b. Everytime we lookup against the REF table we do it by row by row? Correct?
All responses are greatly appreciated. Thanks!
Our environment- 4 CPUs Solaris.
1. We read from two types of tables - source, ref tables.
Our source table gets loaded with 2k rows everyday and cleaned every 2 months(120k max accumulation in 2months). Our REF tables wouldn't exceed 1k rows.
2. There are two ways to extract from the source table. Which is the best method?
a. Use a table READ method in the ODBC stage and use a filter stage.
b. Write a query with a WHERE clause and use User defined SQL.
Our WHERE clause consists of only one column with a status code =0/1 check.
3. During the lookups again Read Method = Table vs User def SQL.
a. Does a Read method = table cache the table into memory and avoid a lookup against the table row by row?
b. Everytime we lookup against the REF table we do it by row by row? Correct?
All responses are greatly appreciated. Thanks!