Partitioning and Collecting

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
rajeevm
Participant
Posts: 135
Joined: Sun Jan 22, 2006 10:44 am

Partitioning and Collecting

Post by rajeevm »

Hi All,

I am facing a problem with the partitioning/collecting method in datastage. I have a job which has two columns Priority_Cd and Priority Order . I have to aggregate count of rows for priority coming from three different files and I have manually derived the priority order column 1 coming from the first file and the second file has 2 and third file has 3 . I used Aggregator stage in order to aggregate count of rows from all the three files and used a funnel stage to get the rows from the three files and should be written to another sequential file(Target). But my Target sequential file should look like this based upon the priority order , the counts should be in the order

File1_Counts,Priority_cd
File2_Counts,Priority_cd
File3_Counts,Priority_cd

For this I made the collector type as 'Ordered' in the target sequential file but its not working. I used the Hash partitioning in all the three Aggregator stages .

I have no idea where exactly my problem. I tried different ways of changing partitioning method /collector types but did not work out.

I appreciate your help.

Thanks
Raj
Kryt0n
Participant
Posts: 584
Joined: Wed Jun 22, 2005 7:28 pm

Post by Kryt0n »

Just sort based on priority... may be better off with a server job too to ensure only one partition.

As for actually collecting in order, what is your link order?
Post Reply