Run time column propagation

dineshrk · Post by **dineshrk** » Tue May 25, 2010 12:23 am

Hi,

I have a job which extract data from SQL db and for this i have enabled RCP and extracted data into a dataset. I vave enabled RCP at stage level and i had defined some column definition which is not part of source metadata. Please let me know how the priority goes here as i have enabled RCP and included some metadata for a file.

I tried running a job but i am not able to find any difference.

Source (ODBC) ----------------------->Tx----------------------->Dataset

Thanks

chulett · Post by **chulett** » Tue May 25, 2010 5:11 am

Please let us know what you mean by "how the priority goes here", I am unclear what it is you are asking or what "difference" you are looking for.

dineshrk · Post by **dineshrk** » Tue May 25, 2010 11:40 pm

Here what i mean is, whether the file will take run time colums extracted from the source or the metadata which i had defined. The defined metadata is not part of source metadata.

chulett wrote:Please let us know what you mean by "how the priority goes here", I am unclear what it is you are asking or what "difference" you are looking for.

agpt · Post by **agpt** » Tue May 25, 2010 11:44 pm

I guess it should take everything from source + extra columns from metadata.

People, please correct me if I am wrong.

chulett · Post by **chulett** » Tue May 25, 2010 11:50 pm

Why not tell us what you saw when you ran the job, what columns were included in the output? That should go a long ways todays answering your question. Then mix it up, re-run and re-see.

dineshrk · Post by **dineshrk** » Wed May 26, 2010 4:13 am

Hi,

I have seen only the columns which are propagated from source (RCP enabled columns) and it had not included the metadata i had defined.

chulett wrote:Why not tell us what you saw when you ran the job, what columns were included in the output? That should go a long ways todays answering your question. Then mix it up, re-run and re-see.

chulett · Post by **chulett** » Wed May 26, 2010 6:35 am

Well, there you go. Read all about RCP in the Parallel Job Developer Guide, page 26. You are missing a critical piece of information.

agpt · Post by **agpt** » Wed May 26, 2010 9:47 am

In that it says:

"You can define part of your schema and specify that, if your job encounters extra columns that are not defined in the meta data when it actually runs, it will adopt these extra columns and propagate them through the rest of the job."

Then he should get all the columns which are in source as well extra ones from metadata. I am not sure why he is not getting extra ones from metadata

chulett · Post by **chulett** » Wed May 26, 2010 10:11 am

Because he is not using a schema file.

agpt · Post by **agpt** » Wed May 26, 2010 10:25 am

oh ok.... I took schema here in generic sense as in meta data rather than as Schema File and that's why I got confused.

Thanks a lot Craig.