Hi,
I am trying to create a sample job to test sort funnel property in the funnel stage.
1st source file
coulmn1
2
6
4
2nd file
column1
2
3
7
I am expecting data to be sorted in ascending order and my final file should be
2
2
3
4
6
7
But i got in different order. I added a environmental variable $apt_Grid_partitions and set to 1 then i got correct order but if i change it to 4 then i am not getting in ascending order.
My question is what is $apt_Grid_partitions and why is it influencing the outcome?
sort funnel and apt_grid_partition
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 236
- Joined: Sun Apr 01, 2007 7:41 am
- Location: Michigan
-
- Participant
- Posts: 31
- Joined: Mon Jun 28, 2010 5:15 am
- Location: pune
Re: sort funnel and apt_grid_partition
$APT_GRID_PARTITIONS Specifies the number of partitions for each compute node.
Hence when you specify 4, the data is partitioned on 4 partitions, but when it is 1 all the records go to a single partition. and hence the output is desired output
Hence when you specify 4, the data is partitioned on 4 partitions, but when it is 1 all the records go to a single partition. and hence the output is desired output
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Sorting is performed separately on each partition. If you wish to preserve the overall sorted order when collecting into a sequential operator then you need to specify SortMerge as the collection algorithm.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.