Page 1 of 1

concatenate two fields (dont use a transformer)

Posted: Sun Oct 14, 2007 8:06 pm
by just4u_sharath
i am working with ascential datastage 7.5.2 in enterprise edition. i have to concatenate two fileds but not using a transformerbecause i am creating parallel jobs and i dont want to take help of transformet. i heard we can do that using a row merge. can i concatenate two fields other than using a transformer. Replies will be appreciated

Posted: Sun Oct 14, 2007 8:22 pm
by ArndW
The row merge does something completely different. The correct stage to use would be the modify stage, I am not sure if the default ":" concatenation operator can be used or if you need to call one of the builtin string functions.
There is no reason not to use the Transform stage; the only stage that is generally recommended against using is the BASIC transform stage.

Posted: Sun Oct 14, 2007 10:37 pm
by ray.wurlod
You must use a Transformer stage unless you can perform the required concatentaion at source (during extraction). Modify stage functions can only be applied to a single input argument, which automatically rules out that stage type.

Your only other alternative is to create some kind of custom stage.

Why is it you don't want to use the Transformer stage?

Posted: Sun Oct 14, 2007 11:19 pm
by ArndW
I just got back to a computer with the docs installed and couldn't find a string/conversion function that can be called in a modify stage; so you will need to use a transform stage to do this.

Posted: Mon Oct 15, 2007 2:05 am
by ameyvaidya
A crazier solution that might just work (cannot test it as no PX available nor have i had to try it out before as i always used a transformer :wink: )is to use a Column Export stage to get the 2 (Or more) columns together in the format needed.

That said, I'd go with the transformer.

Re: concatenate two fields (dont use a transformer)

Posted: Mon Oct 15, 2007 3:26 am
by AKUMAR21
Use of column export stage is very much possible in Px edition since I'm using one. Sharath, if your aim is only to concatenate two or more fields into one field then column export is preffered over transformer.

Posted: Mon Oct 15, 2007 8:49 am
by ray.wurlod
Why?

Posted: Sun Mar 16, 2008 11:58 pm
by AKUMAR21
Simply because using transformer is a costly process eating up a lot of resources and should only be selected for mutlipurpose utility i.e. if you need to do some transformations, merge columns, add some extra columns in the output, reject based on validation failure; all this at the same time then go for the universal stage i.e the transformer. Otherwise if you need to use only one particular functionality then go for other stages. This is why the mutliple functionalities of the transformer has been segregated into various stages in the px. For eg.

1. Simply Map columns from input to output : Copy stage
2.To segeregate records based on certain conditions : Filter stage
3. To concatenate mutliple input columns : Column export stage
4. Split the input column into many columns : Column import stage
......and so on

Posted: Mon Mar 17, 2008 2:01 am
by ArndW
Akumar - I just wrote two small test programs to see what the differences are. In a 1-node configuration my transform stage was 20% faster than a column import stage.

Posted: Mon Mar 17, 2008 2:45 am
by ray.wurlod
AKUMAR21 wrote:Simply because using transformer is a costly process eating up a lot of resources and should only be selected for mutlipurpose utility i.e. if you need to do some transformations, merge columns, add some extra columns in the output, reject based on validation failure; all this at the same time then go for the universal stage i.e the transformer. Otherwise if you need to use only one particular functionality then go for other stages. This is why the mutliple functionalities of the transformer has been segregated into various stages in the px. For eg.

1. Simply Map columns from input to output : Copy stage
2.To segeregate records based on certain conditions : Filter stage
3. To concatenate mutliple input columns : Column export stage
4. Split the input column into many columns : Column import stage
......and so on
Your information was true prior to version 7.5. In version 7.5 they put a lot of work into making the Transformer stage efficient.
Nowadays, and depending on various factors, the Transformer stage can out-perform the Filter stage in evaluating filtering expressions.
But never believe what you read - test it!

Posted: Mon Mar 17, 2008 6:43 am
by chulett
AKUMAR21 wrote:Simply because using transformer is a costly process eating up a lot of resources and should only be selected for mutlipurpose utility
As noted, this information is out-of-date with the advent of the 7.5 version of PX.