I am aggregating/grouping on the first 4 keys to avoid the duplicate value
but for some reason I am getting a duplicate rows. Some rows are eliminted and some are still there.
In the O/P tab I have specified the grouping on first 4 columns
Duplicate record
1026|1024|1000|1010|82|2004-04-27 12:48:33
1026|1024|1000|1010|82|2004-04-27 12:48:33
1026|1024|1000|1010|82|2004-04-27 12:48:33
1006|1022|1000|1010|0|2004-04-27 16:01:27
1006|1022|1000|1010|0|2004-04-27 16:01:27
Does somebody can give some advise.
-Thanks
Aggregator stage
Moderators: chulett, rschirm, roy
Hi,
try to see if there is no other column that has the group by checked on it in any other column/s of your table definition.
IHTH,
try to see if there is no other column that has the group by checked on it in any other column/s of your table definition.
IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
![Image](http://www.worldcommunitygrid.org/images/logo.gif)
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
![Image](http://www.worldcommunitygrid.org/images/logo.gif)
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Ray,
I have only grouping on those 4 columns and they are the Primary key in table which I am trying to insert through SQL loader.
Roy,
I am using SQL loader to load the data in database but in the aggregator I load the column information from table defination. I have seen that the table defination is in sink with aggregator.
The reason why I have this aggregator is to avoid duplicate value. If their is some other way to achive it, do let me know
Thanks for your help.
Please advise.
I have only grouping on those 4 columns and they are the Primary key in table which I am trying to insert through SQL loader.
Roy,
I am using SQL loader to load the data in database but in the aggregator I load the column information from table defination. I have seen that the table defination is in sink with aggregator.
The reason why I have this aggregator is to avoid duplicate value. If their is some other way to achive it, do let me know
Thanks for your help.
Please advise.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
If the input data are sorted on the four grouping columns, you can use stage variables to detect change. This is a much more efficient mechanism than the Aggregator stage. Search the Forum for examples.
Another approach is to use a hashed file keyed on these four columns. All writes to hashed files are destructive overwrites (based on key) so it's a fairly slick means of de-duplication.
Another approach is to use a hashed file keyed on these four columns. All writes to hashed files are destructive overwrites (based on key) so it's a fairly slick means of de-duplication.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.