Sort stage is not working
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 111
- Joined: Thu Jun 01, 2006 5:12 am
- Location: Detroit
Sort stage is not working
Hi,
I am using sort stage in my parallel job once read data by using sequential file next i am using sort stage, it's not sorting as i given and send ing same input records to output so that ia m facing problem and caluculation is not happening in next transformer stage,
so pls any one has experience with sort stage with this type of problems....
Actually the number of records in file is 1million, is there any problem with these amny records , any body have on this.......pls share
I am using sort stage in my parallel job once read data by using sequential file next i am using sort stage, it's not sorting as i given and send ing same input records to output so that ia m facing problem and caluculation is not happening in next transformer stage,
so pls any one has experience with sort stage with this type of problems....
Actually the number of records in file is 1million, is there any problem with these amny records , any body have on this.......pls share
kOTI,
1586-738-7785
1586-738-7785
-
- Participant
- Posts: 111
- Joined: Thu Jun 01, 2006 5:12 am
- Location: Detroit
Bollenik - most likely you are using a multi-node configuration and not accounting for how PX works. If you change to a 1-node configuration do you get the results you are looking for?
Last edited by ArndW on Wed Feb 28, 2007 7:58 am, edited 1 time in total.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 111
- Joined: Thu Jun 01, 2006 5:12 am
- Location: Detroit
Hi novneet,
Earlier I am doing sorting, once I read data next stage doing sorting, So after all lookup check's and all doing some caluculation so that as per pallel partitioning and all output is not coming properly. So, now i am doing sorting just before the caluculation that means after all lokup cheks and all. and also I asm unsing same partition to continue the partition block. so that output is ok now
Earlier I am doing sorting, once I read data next stage doing sorting, So after all lookup check's and all doing some caluculation so that as per pallel partitioning and all output is not coming properly. So, now i am doing sorting just before the caluculation that means after all lokup cheks and all. and also I asm unsing same partition to continue the partition block. so that output is ok now
kOTI,
1586-738-7785
1586-738-7785
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The main point - which you don't show that you understand - is that for sorting to work properly in parallel the partitioning must ensure that each key value occurs only on one partition.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.