Parallel Pivot and Build-ops

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
vijayrc
Participant
Posts: 197
Joined: Sun Apr 02, 2006 10:31 am
Location: NJ

Parallel Pivot and Build-ops

Post by vijayrc »

Hi,
We started using Pivot [though not in parallel mode] and was advised to go for build-ops, which would be better in performance. But of late, realised that Pivot Stage can be used in Parallel mode and it looks like the performance is in par with that and infact slightly better than that of build-ops [though testing is going on to further justify Pivot stage].
I'm trying to get any first hand information from ppl out here who had done this research [Pivot vs Build Op] and any thoughts is highly appreciated.
The Build Ops we have are very simple, and just basic moves or minimal arithmetics...fyi
Thanks in advance
Vijay
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

BuildOp gives you total control over the C++ code, and therefore over the code optimization. In theory, therefore, you should be able to "build a better mousetrap"; in your case a stage that outperforms the Pivot stage, provided that you have the skills to optimize code.

If you don't, why re-invent the wheel? Use the Pivot stage.

Make very sure that your data are correctly key partitioned and sorted.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply