Need a clarification
Posted: Thu Jun 23, 2011 10:23 pm
Hi All,
I need this clarification from all of you. Thanks in advance. :D
My job design is:
Dataset -----> Transformer ------> Remove Duplicate ------> Dataset
In the input link of the transformer stage, I am performing a 'HASH' partioning.
In the input link of the Remove Duplicate I am using 'SAME' partioning.
Now, my question I got the information from my code reviewer that transformer stage is not capable of retening partioning in the output link.
It automatically converts the partioning to 'AUTO'. So, RD stage will have 'AUTO' partioning in the input link. Is that so?
2. RD is a key based stage, it is getting 'AUTO' partioning the input link (if my question 1 is correct). So, will DS optimize the partioning to 'HASH'?
I will be delighted if you all can throw some light on my doubts.
I need this clarification from all of you. Thanks in advance. :D
My job design is:
Dataset -----> Transformer ------> Remove Duplicate ------> Dataset
In the input link of the transformer stage, I am performing a 'HASH' partioning.
In the input link of the Remove Duplicate I am using 'SAME' partioning.
Now, my question I got the information from my code reviewer that transformer stage is not capable of retening partioning in the output link.
It automatically converts the partioning to 'AUTO'. So, RD stage will have 'AUTO' partioning in the input link. Is that so?
2. RD is a key based stage, it is getting 'AUTO' partioning the input link (if my question 1 is correct). So, will DS optimize the partioning to 'HASH'?
I will be delighted if you all can throw some light on my doubts.