Sort stage is not working

bollinenik · Post by **bollinenik** » Wed Feb 28, 2007 3:45 am

Hi,

I am using sort stage in my parallel job once read data by using sequential file next i am using sort stage, it's not sorting as i given and send ing same input records to output so that ia m facing problem and caluculation is not happening in next transformer stage,
so pls any one has experience with sort stage with this type of problems....

Actually the number of records in file is 1million, is there any problem with these amny records , any body have on this.......pls share

novneet · Post by **novneet** » Wed Feb 28, 2007 3:51 am

Is there anykind of warning/error message comming in the director?

bollinenik · Post by **bollinenik** » Wed Feb 28, 2007 4:07 am

yes there are warnings but not for that stage, some other lookup and all giving warnings, there are no warnings for sort and sequential file stage, but it's not sorting data properly,
share your ideas with me

novneet · Post by **novneet** » Wed Feb 28, 2007 4:12 am

can you please give a tentative structure of the job and what is the warning which is comming in the director.

ArndW · Post by **ArndW** » Wed Feb 28, 2007 4:31 am

Bollenik - most likely you are using a multi-node configuration and not accounting for how PX works. If you change to a 1-node configuration do you get the results you are looking for?

kumar_s · Post by **kumar_s** » Wed Feb 28, 2007 7:54 am

Arnd is focusing on the partition method you have used. Improper partition method will give you wrong result.
To get a better picture, you can explain on what is expected and what is the actual outcome.

novneet · Post by **novneet** » Thu Mar 01, 2007 1:14 am

can you please post the details how the problem resolved.

bollinenik · Post by **bollinenik** » Thu Mar 01, 2007 3:12 am

Hi novneet,

Earlier I am doing sorting, once I read data next stage doing sorting, So after all lookup check's and all doing some caluculation so that as per pallel partitioning and all output is not coming properly. So, now i am doing sorting just before the caluculation that means after all lokup cheks and all. and also I asm unsing same partition to continue the partition block. so that output is ok now

ray.wurlod · Post by **ray.wurlod** » Thu Mar 01, 2007 3:51 pm

The main point - which you don't show that you understand - is that for sorting to work properly in parallel the partitioning must ensure that each key value occurs only on one partition.