Use of recalculation in aggregator stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Use of recalculation in aggregator stage

Post by zulfi123786 »

What is the advantage of the option "recalculation" in the aggregator stage ?
Looks like it does the same thing as normal aggregation, is it only to avoid repartitioning and resorting on the same keys ?
- Zulfi
felixyong
Participant
Posts: 35
Joined: Tue Jul 22, 2003 7:24 pm
Location: Australia

Re: Use of recalculation in aggregator stage

Post by felixyong »

Look at Pg 224 of Parallel Job Developer Guide for V8.5 give you a good example on the usage of Re-calculation.
Regards
Felix
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Post by zulfi123786 »

Thanks for the reference but I don't have 8.5 mannuals yet, still running on 8.1. Is the same documented in 8.1 ?
I have definitely read an example in the guide of 8.1 (not sure if it's the same example you are referring to), it demonstrates how recalculation should be implmented but my question is "from what is documented it's clear that recalculation could be achieved using calculation mode in the following aggregator then what benefits does recalculation offer over to calculation mode"
- Zulfi
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

"Re-calculation allows you to apply aggregate functions to a column that has already been summarized." - Parallel Job Developer's Guide
Choose a job you love, and you will never have to work a day in your life. - Confucius
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Post by zulfi123786 »

qt_ky wrote:"Re-calculation allows you to apply aggregate functions to a column that has already been summarized." - Parallel Job Developer's Guide
Agreed ! but why is there a need to provide an option of "recalculate" in the downstream aggregator stage when the same can be achieved using calculate option of the downstream aggregator stage ?
- Zulfi
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

It's implied that it will save on processing. My imagination tells me that suppose you were tracking some total count and every day you had new data to add to the count, you could use recalculate to add the delta rather than calculating all days every day.
Choose a job you love, and you will never have to work a day in your life. - Confucius
pandeesh
Premium Member
Premium Member
Posts: 1399
Joined: Sun Oct 24, 2010 5:15 am
Location: CHENNAI, TAMIL NADU

Post by pandeesh »

qt_ky wrote: My imagination tells me that suppose you were tracking some total count and every day you had new data to add to the count, you could use recalculate to add the delta rather than calculating all days every day.
Your imagination seems good. But i am not able to imagine how we can implement this in real time using recalculation. :)
pandeeswaran
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Think about the possibility of updating shrunken (summarised) fact tables.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply