Duplicates after pivot stage

vinodhraj · Post by **vinodhraj** » Tue Nov 28, 2006 7:26 am

Hi,

I am facing duplicates once the data crosses the pivot stage. Is there any solution to prevent in pivot stage itself.

Thanks

Vinod

chulett · Post by **chulett** » Tue Nov 28, 2006 8:02 am

Post some examples, before and after. The pivot stage really isn't about de-duping but examples would help us understand your situation.

vinodhraj · Post by **vinodhraj** » Tue Nov 28, 2006 8:16 am

hi Creig,

For eg:

input

item_no, week_id, version1,version2,version3,version4,version5

output

item_no, week_id, version

1 1 version1
2 2 version2
3 3 version3
4 4 version4
5 5 version5

here after pivot stage , I am getting 4 duplicates excluding the first one.
Then the value of 4 duplicates were null for the version column alone.

Thanks

Vinod

chulett · Post by **chulett** » Tue Nov 28, 2006 8:24 am

Did you mark any fields as Keys in the Pivot stage? It looks like you should have three fields in the stage and the first two should be marked as keys. The third one, with a derivation of "version1,version2,version3,version4,version5", should not be a key field.

vinodhraj · Post by **vinodhraj** » Tue Nov 28, 2006 8:46 am

Thanks Craig.

Even though I have not mentioned as key, I am facing the same issue.

Thanks

Vinod

chulett · Post by **chulett** » Tue Nov 28, 2006 9:15 am

No, you are facing 'the same issue' precisely because you have 'not mentioned as key'. Mark the first two fields as keys and try it again! Or is that a typo in your post?