Page 1 of 1
improving the performance in transformer stage
Posted: Wed Sep 08, 2010 12:28 am
by mallikharjuna
Hi All,
My Source table is having 50 lacks records.i have applied few transformation rules in trasformer stage based on my requirements.my job is taking 40 minutes time to finish.
please let me know how to improve the performace in transformer stage.
Thanks in advance
Posted: Wed Sep 08, 2010 1:05 am
by udayk_2007
What kind of calculations/processing are you doing in transformer stage ?
Posted: Wed Sep 08, 2010 1:31 am
by ETLJOB
Are you sure, the bottle neck is in transformer? What are the other stages you have used in the job? Are the performance statistics of the other stages looking good? Post your analysis please!
Posted: Wed Sep 08, 2010 2:01 am
by ray.wurlod
Look for any opportunity for doing less processing, for example evaluating functions once in stage variables rather than many times in column derivation expressions, evaluating constants once when initializing stage variables rather than once per row, and so on. Once we see what you're doing more suggestions may be forthcoming.
Posted: Thu Sep 09, 2010 5:03 am
by natashab
Which stage are you using to extract the data?If it's Oracle OCI stage then the array size can be set to the maximun value(32767) to improve the performance of the job