Performance tuning
Posted: Fri Jan 26, 2007 1:13 am
Hi Gurus,
I need some advice on performance tuning in datastage Parallel extender.
This is the scenario.
1. I am using teradata enterprise stage to read a table and do some processing using lookup stage, sort stage and aggregator stage and finally loading 30 million records into the teradata table using teradata multiload stage. (runtime is 40 min)
2. To reduce run time, I just tried to do the sorting and aggregation in teradata enterprise stage itself and then do the same lookups, then loading into teradata table using teradata multiload stage - 30 million records. (runtime is 39 min).
My assumption was the scenario 2 should run very fast when compared to scenario 1. But it was not. Why is that ? Can some one give an explanation about this ? Is my assumption wrong? ( actually I expected the scenario 2 will run in 20 or 25 min, since it avoids sort and aggregate stage in datastage)
Thanks in advance
I need some advice on performance tuning in datastage Parallel extender.
This is the scenario.
1. I am using teradata enterprise stage to read a table and do some processing using lookup stage, sort stage and aggregator stage and finally loading 30 million records into the teradata table using teradata multiload stage. (runtime is 40 min)
2. To reduce run time, I just tried to do the sorting and aggregation in teradata enterprise stage itself and then do the same lookups, then loading into teradata table using teradata multiload stage - 30 million records. (runtime is 39 min).
My assumption was the scenario 2 should run very fast when compared to scenario 1. But it was not. Why is that ? Can some one give an explanation about this ? Is my assumption wrong? ( actually I expected the scenario 2 will run in 20 or 25 min, since it avoids sort and aggregate stage in datastage)
Thanks in advance