RCP Job - Dynamic Concatenation

[koti9]

Hello all,

We are trying to dynamically concatenate columns(Column names can be parameterised) and so is the output column in an RCP Enabled job.

I must use RCP. Thats the only transformation i need to do.

Any inputs are greatly appreciated..

Thanks
Koti

[ray.wurlod]

Any transformation needs to name the input column(s). That's an absolute rule!

You can't use a Modify stage here, since concatenation is not available as a Modify stage operator/function.

[rkashyap]

While this post did not clearly describe your motive, based on the previous post, I am assuming that you are trying to concatenate keys to support CDC.

As Ray has stated, named columns is a must for concatenation.

A workaround for this can be designed by using Column Export stage.
1. Define Schema files (one per table) listing the columns that need to be concatenated.
2. Parameterize schema-filename and pass it to a generic job described below.
3. Create job using Column Export stage to concatenate columns, passing schema-filename in properties.

[ArndW]

Since your column names are parameterized, you can use a MODIFY stage to "name" your parameterized columns, i.e.

Code: Select all

TEMPCOLUMN1=#ParameterColumnName1#
TEMPCOLUMN2=#ParameterColumnName2#

Then do your concatentation to a newly created column a in a subsequent Transform stage using your 2 columns.

If you need those 2 columns in the output, then you need another modify downstream of the transform that does

Code: Select all

ParameterColumnName1#=TEMPCOLUMN1#
ParameterColumnName2#=TEMPCOLUMN2#

[koti9]

Thanks Ray,Kashyap,ArndW..

Kashyap, for Column Export Stage, output column name cannot be parameterised..though input columns can be parameterised...


Thanks
Koti

[rkashyap]

Koti, Use a generic output column-name in Column Export stage. Rename it in MODIFY stage to the parameterized column-name.

[koti9]

Thanks Kashyap, it worked..i don't know why i dint think of modify stage to rename the column dynamically..

Thanks
Koti