DSXchange

I have a modify stage in my job. All input column are populated. However, the output columns are empty. I want to move all input columns automatically to output columns without having to import it from somewhere. How do I do it?

You set up Specifications.

Each output column must have a specification.

You can have a KEEP specification to pass through column(s) that do not need to be modified, or a DROP specification to manage column(s) that do not need to be passed through, but not both.

Every other output column needs a Specification. You will note, from its icon, that Specification is a multiple property.

You might also read vmcburney's blog on Learning the Modify Stage

ray, thank you for your response. I know about the KEEP specification. That requires that I specify every column that I need to keep. I have over 100 columns. I was wondering if there was another way or some other technique to get around it and apprently there isn't.

Thanks.

Actually, I wanted the columns from input tab moved to the output tab at design time. It doesn't happen. How do I get the columns moved at design time without manually typing them?

abc123 wrote:Actually, I wanted the columns from input tab moved to the output tab at design time. It doesn't happen. How do I get the columns moved at design time without manually typing them?

enable runtime column propagation for the job.

Sree

No, it doesn't work. I enabled RCP across the project, job and stage. I still see the Output tab as empty. There is no way to drag the columns from the input tab to the output tab.

I don't know what exactly you are trying to do. But with your RCP enabled, You can just use DROP option to drop the columns which are not needed and this will retain the other columns. whereas in KEEP option you need to specify every single column that is needed in the output.

Sam

abc123 wrote:No, it doesn't work. I enabled RCP across the project, job and stage. I still see the Output tab as empty.

Of course it stays empty - that's why it's called Runtime Column Propagation. It happens automatically at runtime.

Well. After the modify stage, I have a sequential file stage. The input of this stage has no columns because the output of the modify stage has no columns. The job will not compile. I had to do this manually. Took a lot of time.

As I commented on Vincent's blog, the Modify stage is primitive. Deliberately. That's the secret of its success - it's fast, because YOU do the work.

Seems obvious which leads me to believe I am missing the point. Can you do a save of the input tab of the modify stage into a table definition and then do a load into the output tab of the same table definition? It appears your problem is that there is no easy way to drag and drop the link columns from the input to the output column of the modify stage. Doing it via RCP is an option but unless you specifcally want RCP functionality, I would avoid it since as you see it hides important Metadata details like the actual columns.