Max Value from each partition

r_arora · Post by **r_arora** » Mon Apr 07, 2008 10:17 am

Hello,

I have a scenario where I want the maximum date from each partition.And from the set of rows I get, I have to get the maximum of that date and write it to a dataset. For example, If I have 4 partitions I will get 4 max dates, then I will sort these set of 4 dates and get the maximum out of these 4.

My source is about 9 million records. And I need to process only these 4 records to get the MAXDATE after the transformer from these 9 million records.
My question is, how will I know the number of partitions and how will I get the maximum value from each partition.
Any help will be appreciated.
Thanks

ray.wurlod · Post by **ray.wurlod** » Mon Apr 07, 2008 3:25 pm

An Aggregator stage or a Transformer stage can give you the maximum from each partition.

Nripendra Chand · Post by **Nripendra Chand** » Mon Apr 07, 2008 6:14 pm

you can use '@NUMPARTITIONS' system variable in transformer stage to know the number of partitions.