Page 1 of 1

Parallel Pivot and Build-ops

Posted: Mon Jan 29, 2007 6:58 pm
by vijayrc
Hi,
We started using Pivot [though not in parallel mode] and was advised to go for build-ops, which would be better in performance. But of late, realised that Pivot Stage can be used in Parallel mode and it looks like the performance is in par with that and infact slightly better than that of build-ops [though testing is going on to further justify Pivot stage].
I'm trying to get any first hand information from ppl out here who had done this research [Pivot vs Build Op] and any thoughts is highly appreciated.
The Build Ops we have are very simple, and just basic moves or minimal arithmetics...fyi
Thanks in advance
Vijay

Posted: Mon Jan 29, 2007 7:53 pm
by ray.wurlod
BuildOp gives you total control over the C++ code, and therefore over the code optimization. In theory, therefore, you should be able to "build a better mousetrap"; in your case a stage that outperforms the Pivot stage, provided that you have the skills to optimize code.

If you don't, why re-invent the wheel? Use the Pivot stage.

Make very sure that your data are correctly key partitioned and sorted.