Selective reading Metadata versus using a copy stage/Modify
Posted: Mon Dec 10, 2007 8:49 am
Hi Folks,
Here is my dilemma:
I have 200 columns in the input source and I need only 100 for downstream processing, Should I read all 200 with a dataset and drop redundant using a copy/modify stage as next step or should I restrict the metadata definition in the input dataset to read only 100 selective columns (by altering the column definition/layout)
I am looking from the code maintainability and most importantly performance point of view.
Appreciate your help.
Here is my dilemma:
I have 200 columns in the input source and I need only 100 for downstream processing, Should I read all 200 with a dataset and drop redundant using a copy/modify stage as next step or should I restrict the metadata definition in the input dataset to read only 100 selective columns (by altering the column definition/layout)
I am looking from the code maintainability and most importantly performance point of view.
Appreciate your help.