DSXchange

Hi,

I am facing duplicates once the data crosses the pivot stage. Is there any solution to prevent in pivot stage itself.

Thanks

Vinod

Post some examples, before and after. The pivot stage really isn't about de-duping but examples would help us understand your situation.

hi Creig,

For eg:

input

item_no, week_id, version1,version2,version3,version4,version5

output

item_no, week_id, version

1 1 version1
2 2 version2
3 3 version3
4 4 version4
5 5 version5


here after pivot stage , I am getting 4 duplicates excluding the first one.
Then the value of 4 duplicates were null for the version column alone.


Thanks

Vinod

Did you mark any fields as Keys in the Pivot stage? It looks like you should have three fields in the stage and the first two should be marked as keys. The third one, with a derivation of "version1,version2,version3,version4,version5", should not be a key field.

Thanks Craig.

Even though I have not mentioned as key, I am facing the same issue.

Thanks

Vinod

No, you are facing 'the same issue' precisely because you have 'not mentioned as key'. Mark the first two fields as keys and try it again! Or is that a typo in your post?