Pivot perfomance is bad

santoshkumar · Post by **santoshkumar** » Tue Dec 05, 2006 11:50 am

Hi,

I am running a job in DS7.5.1A and it is taking for 0.6 million records 3 hours.

Design:

oracle source--->Xfm------>Pivot----------->DataSet.

From oracle stage to Xfm it is taking only 3 minutes.

Pivot stage is the overhead.

How can i improve the performance.
I am pivoting 16 columns to rows.

Thanks.

thebird · Post by **thebird** » Tue Dec 05, 2006 11:57 am

santoshkumar wrote: Pivot stage is the overhead.

The pivot stage is known to give a performance overhead, especially when pivoting too many columns.

By default the Pivot stage runs in a Sequential mode - have you changed this to run in a parallel mode? This should improve the performance a little bit.

Aneesh

santoshkumar · Post by **santoshkumar** » Tue Dec 05, 2006 1:28 pm

Is there any other way i can replicate the pivot stage functionality and improve performance.

thebird · Post by **thebird** » Tue Dec 05, 2006 1:50 pm

santoshkumar wrote:Is there any other way i can replicate the pivot stage functionality and improve performance.

Did you try to run in a Parallel mode? Any significant difference?

santoshkumar · Post by **santoshkumar** » Tue Dec 05, 2006 4:20 pm

Not much of diff.

I am just thinking of having 8 output links form tranformer to pivot with each having pivot columns.

I think this should give me some improvement.

If anyone has any other thought just let me know..

thebird · Post by **thebird** » Tue Dec 05, 2006 4:22 pm

santoshkumar wrote:I am just thinking of having 8 output links form tranformer to pivot with each having pivot columns.

Why dont you try writing the Pivot Logic inside the Transformer and checkout the performance?

santoshkumar · Post by **santoshkumar** » Tue Dec 05, 2006 9:22 pm

how can i achieve in a transformer

DSXchange

Pivot perfomance is bad

Pivot perfomance is bad

Re: Pivot perfomance is bad