-------Rows missing in tha Hash File--------
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 9
- Joined: Sat Jul 22, 2006 5:40 am
-------Rows missing in tha Hash File--------
Hi,
We have developed a job which sorts records on the basis of a field.
Later on, these records are loaded into a hash file.
The Problem what I am facing is, Some records are missing when i start searching in the hash file. I connected a Sequential file instead of the Hash file and found out that all the records are correct...
When I run the job, the performance statistics says 20793 records are loaded into the Hash file as well as the Sequential file. But , The records loaded into the Hash file are less...The records in the Sequential and Hash file are different in the sense that, some records are missing from the Hash File. I need to Use The Hash File of this job for the next Job.
Could anyone help me on this issue...
Thank you.
Alan
We have developed a job which sorts records on the basis of a field.
Later on, these records are loaded into a hash file.
The Problem what I am facing is, Some records are missing when i start searching in the hash file. I connected a Sequential file instead of the Hash file and found out that all the records are correct...
When I run the job, the performance statistics says 20793 records are loaded into the Hash file as well as the Sequential file. But , The records loaded into the Hash file are less...The records in the Sequential and Hash file are different in the sense that, some records are missing from the Hash File. I need to Use The Hash File of this job for the next Job.
Could anyone help me on this issue...
Thank you.
Alan
-
- Participant
- Posts: 9
- Joined: Sat Jul 22, 2006 5:40 am
-
- Participant
- Posts: 9
- Joined: Sat Jul 22, 2006 5:40 am
-
- Participant
- Posts: 9
- Joined: Sat Jul 22, 2006 5:40 am
Thank you very much..
I got through the problem..
Actually, I had a doubt. The job is like this.
Skey Skey
Amount---->Aggregator (Sum on amount)-----> New_Amount
EID EID
AE Skey AE Skey
After this I need to pick up the records with max(New_Amount) Corresponding to each Key. I Put a Aggregator stage but it was not helping me get the correct result...
I Put the Sort stage after the Aggregator(Summation) and specified the properties(Sort Specification) as New_Amount ASC . I Did get the Max Amount in the Hash File which was the Target. But, Actually in this case What I feel is that,the lowest amount should have been loaded for the corresponding key specified in the hash
file When the records are overided....I'm not sure of the result...
I got through the problem..
Actually, I had a doubt. The job is like this.
Skey Skey
Amount---->Aggregator (Sum on amount)-----> New_Amount
EID EID
AE Skey AE Skey
After this I need to pick up the records with max(New_Amount) Corresponding to each Key. I Put a Aggregator stage but it was not helping me get the correct result...
I Put the Sort stage after the Aggregator(Summation) and specified the properties(Sort Specification) as New_Amount ASC . I Did get the Max Amount in the Hash File which was the Target. But, Actually in this case What I feel is that,the lowest amount should have been loaded for the corresponding key specified in the hash
file When the records are overided....I'm not sure of the result...
Hi
I don't understand your last post.
If you sum your amount on Skey you will have only one row in the aggregator ouput.
Skey Amount
1 100
2 200
1 100
2 200
3 300
with this data the result will be
Skey NewAmount
1 200
2 400
3 300
And now the hash_file with don't override any records.
I don't understand your last post.
If you sum your amount on Skey you will have only one row in the aggregator ouput.
Skey Amount
1 100
2 200
1 100
2 200
3 300
with this data the result will be
Skey NewAmount
1 200
2 400
3 300
And now the hash_file with don't override any records.
Hope This Helps
Regards
Regards
-
- Participant
- Posts: 9
- Joined: Sat Jul 22, 2006 5:40 am
The values are like this
Skey Amount EID AESkey
1 20 35 12
2 30 45 23
1 5 35 12
1 13 12 15
The output from the Aggregator After Summation on Amount With Group By function on rest Three Fields will give
Skey Amount EID AESkey
1 25 35 12
2 30 45 23
1 13 12 15
Now I need to Pick the MAX(Amount ) Based on the Skey.
I hope this will help you understand the problem..
Thank You
Skey Amount EID AESkey
1 20 35 12
2 30 45 23
1 5 35 12
1 13 12 15
The output from the Aggregator After Summation on Amount With Group By function on rest Three Fields will give
Skey Amount EID AESkey
1 25 35 12
2 30 45 23
1 13 12 15
Now I need to Pick the MAX(Amount ) Based on the Skey.
I hope this will help you understand the problem..
Thank You
I think that you can't do that without lose or separate the Skey.
I think a transformer with an aggregator will do what you want but again a key is a key if it's unique.
Agg1(Sum(amount) on Skey,EID,Aeskey) ---> Transformer (Skey,NewAmount) ---> Agg2 (Max(amount) on Skey) ---> hash_file (Key?)
I don't know if my i'm clear.
I think a transformer with an aggregator will do what you want but again a key is a key if it's unique.
Agg1(Sum(amount) on Skey,EID,Aeskey) ---> Transformer (Skey,NewAmount) ---> Agg2 (Max(amount) on Skey) ---> hash_file (Key?)
I don't know if my i'm clear.
Hope This Helps
Regards
Regards
-
- Participant
- Posts: 1
- Joined: Fri Oct 21, 2005 4:47 am
hi
Just Sort the file based on SKEY and then sort numeric reverse based on Amount .. Then choose the first incoming record for SKEY and that will be the one with MAX amount....
-
- Participant
- Posts: 9
- Joined: Sat Jul 22, 2006 5:40 am
Re: hi
Nirah,
Can you please elaborate on this..
Can you please elaborate on this..
-
- Participant
- Posts: 9
- Joined: Sat Jul 22, 2006 5:40 am
Re: hi
I found the solution!
Put two aggregators, one for the sum and the other for the max. Then put a sorter in ascending and load it to the hash file. This will help get the required output :D
Put two aggregators, one for the sum and the other for the max. Then put a sorter in ascending and load it to the hash file. This will help get the required output :D