Moving columns from input tab to output tab

abc123 · Post by **abc123** » Wed Dec 20, 2006 4:19 pm

I have a modify stage in my job. All input column are populated. However, the output columns are empty. I want to move all input columns automatically to output columns without having to import it from somewhere. How do I do it?

ray.wurlod · Post by **ray.wurlod** » Wed Dec 20, 2006 5:22 pm

You set up Specifications.

Each output column must have a specification.

You can have a KEEP specification to pass through column(s) that do not need to be modified, or a DROP specification to manage column(s) that do not need to be passed through, but not both.

Every other output column needs a Specification. You will note, from its icon, that Specification is a multiple property.

You might also read vmcburney's blog on Learning the Modify Stage

abc123 · Post by **abc123** » Thu Dec 21, 2006 9:08 am

ray, thank you for your response. I know about the KEEP specification. That requires that I specify every column that I need to keep. I have over 100 columns. I was wondering if there was another way or some other technique to get around it and apprently there isn't.

Thanks.

abc123 · Post by **abc123** » Thu Dec 21, 2006 9:37 am

Actually, I wanted the columns from input tab moved to the output tab at design time. It doesn't happen. How do I get the columns moved at design time without manually typing them?

s_boyapati · Post by **s_boyapati** » Thu Dec 21, 2006 10:11 am

abc123 wrote:Actually, I wanted the columns from input tab moved to the output tab at design time. It doesn't happen. How do I get the columns moved at design time without manually typing them?

enable runtime column propagation for the job.

Sree

abc123 · Post by **abc123** » Thu Dec 21, 2006 10:25 am

No, it doesn't work. I enabled RCP across the project, job and stage. I still see the Output tab as empty. There is no way to drag the columns from the input tab to the output tab.

us1aslam1us · Post by **us1aslam1us** » Thu Dec 21, 2006 10:38 am

I don't know what exactly you are trying to do. But with your RCP enabled, You can just use DROP option to drop the columns which are not needed and this will retain the other columns. whereas in KEEP option you need to specify every single column that is needed in the output.

Sam

chulett · Post by **chulett** » Thu Dec 21, 2006 10:40 am

abc123 wrote:No, it doesn't work. I enabled RCP across the project, job and stage. I still see the Output tab as empty.

Of course it stays empty - that's why it's called Runtime Column Propagation. It happens automatically at runtime.

abc123 · Post by **abc123** » Thu Dec 21, 2006 11:17 am

Well. After the modify stage, I have a sequential file stage. The input of this stage has no columns because the output of the modify stage has no columns. The job will not compile. I had to do this manually. Took a lot of time.

ray.wurlod · Post by **ray.wurlod** » Thu Dec 21, 2006 2:28 pm

As I commented on Vincent's blog, the Modify stage is primitive. Deliberately. That's the secret of its success - it's fast, because YOU do the work.

trobinson · Post by **trobinson** » Fri Dec 22, 2006 7:07 am

Seems obvious which leads me to believe I am missing the point. Can you do a save of the input tab of the modify stage into a table definition and then do a load into the output tab of the same table definition? It appears your problem is that there is no easy way to drag and drop the link columns from the input to the output column of the modify stage. Doing it via RCP is an option but unless you specifcally want RCP functionality, I would avoid it since as you see it hides important Metadata details like the actual columns.