Setting up RCP job
Posted: Mon Mar 03, 2014 11:31 am
We don't have RCP set up for any of our projects or jobs, but since we've got a little downtime, we're interested in understanding the concept...but haven't had much success.
So, in the Administrator I have a test project set to 'Enable Runtime Column Propagation for Parallel jobs'.
In a test job for the same project, I've set 'Enable Runtime Column Propagation for new links'.
My understanding is that with an Oracle Connector stage as the source, we can set this up to be dynamic (variable table name, not provide columns) and that with RCP, the columns of the table name given will ripple through the project.
This seems to work with a straight move from a target to source, but when I look to make a change to one column (in a transformer stage, and only including that column), then just that one column gets written to a target sequential file. I guess I understood that the other columns, not in the transformer, would be written to the sequential file unaltered.
Where is the gap in my knowledge? How do i get the job to take in any table name and write out all of the columns whether I specify a column subset in a middle stage or not?
After I get a successful test case from an Oracle Connector source, I'd like to try using a Sequential File as a source and load a table in an Oracle Connector. I understand that'll require defining part of the schema, which I'll likely have questions on as well.
So, in the Administrator I have a test project set to 'Enable Runtime Column Propagation for Parallel jobs'.
In a test job for the same project, I've set 'Enable Runtime Column Propagation for new links'.
My understanding is that with an Oracle Connector stage as the source, we can set this up to be dynamic (variable table name, not provide columns) and that with RCP, the columns of the table name given will ripple through the project.
This seems to work with a straight move from a target to source, but when I look to make a change to one column (in a transformer stage, and only including that column), then just that one column gets written to a target sequential file. I guess I understood that the other columns, not in the transformer, would be written to the sequential file unaltered.
Where is the gap in my knowledge? How do i get the job to take in any table name and write out all of the columns whether I specify a column subset in a middle stage or not?
After I get a successful test case from an Oracle Connector source, I'd like to try using a Sequential File as a source and load a table in an Oracle Connector. I understand that'll require defining part of the schema, which I'll likely have questions on as well.