improving the performance in transformer stage

mallikharjuna · Post by **mallikharjuna** » Wed Sep 08, 2010 12:28 am

Hi All,

My Source table is having 50 lacks records.i have applied few transformation rules in trasformer stage based on my requirements.my job is taking 40 minutes time to finish.

please let me know how to improve the performace in transformer stage.

Thanks in advance

udayk_2007 · Post by **udayk_2007** » Wed Sep 08, 2010 1:05 am

What kind of calculations/processing are you doing in transformer stage ?

ETLJOB · Post by **ETLJOB** » Wed Sep 08, 2010 1:31 am

Are you sure, the bottle neck is in transformer? What are the other stages you have used in the job? Are the performance statistics of the other stages looking good? Post your analysis please!

ray.wurlod · Post by **ray.wurlod** » Wed Sep 08, 2010 2:01 am

Look for any opportunity for doing less processing, for example evaluating functions once in stage variables rather than many times in column derivation expressions, evaluating constants once when initializing stage variables rather than once per row, and so on. Once we see what you're doing more suggestions may be forthcoming.

natashab · Post by **natashab** » Thu Sep 09, 2010 5:03 am

Which stage are you using to extract the data?If it's Oracle OCI stage then the array size can be set to the maximun value(32767) to improve the performance of the job