concatenate two fields (dont use a transformer)

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
just4u_sharath
Premium Member
Premium Member
Posts: 236
Joined: Sun Apr 01, 2007 7:41 am
Location: Michigan

concatenate two fields (dont use a transformer)

Post by just4u_sharath »

i am working with ascential datastage 7.5.2 in enterprise edition. i have to concatenate two fileds but not using a transformerbecause i am creating parallel jobs and i dont want to take help of transformet. i heard we can do that using a row merge. can i concatenate two fields other than using a transformer. Replies will be appreciated
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The row merge does something completely different. The correct stage to use would be the modify stage, I am not sure if the default ":" concatenation operator can be used or if you need to call one of the builtin string functions.
There is no reason not to use the Transform stage; the only stage that is generally recommended against using is the BASIC transform stage.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You must use a Transformer stage unless you can perform the required concatentaion at source (during extraction). Modify stage functions can only be applied to a single input argument, which automatically rules out that stage type.

Your only other alternative is to create some kind of custom stage.

Why is it you don't want to use the Transformer stage?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I just got back to a computer with the docs installed and couldn't find a string/conversion function that can be called in a modify stage; so you will need to use a transform stage to do this.
ameyvaidya
Charter Member
Charter Member
Posts: 166
Joined: Wed Mar 16, 2005 6:52 am
Location: Mumbai, India

Post by ameyvaidya »

A crazier solution that might just work (cannot test it as no PX available nor have i had to try it out before as i always used a transformer :wink: )is to use a Column Export stage to get the 2 (Or more) columns together in the format needed.

That said, I'd go with the transformer.
Amey Vaidya<i>
I am rarely happier than when spending an entire day programming my computer to perform automatically a task that it would otherwise take me a good ten seconds to do by hand.</i>
<i>- Douglas Adams</i>
AKUMAR21
Participant
Posts: 30
Joined: Fri Jul 20, 2007 12:44 am
Location: Chennai

Re: concatenate two fields (dont use a transformer)

Post by AKUMAR21 »

Use of column export stage is very much possible in Px edition since I'm using one. Sharath, if your aim is only to concatenate two or more fields into one field then column export is preffered over transformer.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Why?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
AKUMAR21
Participant
Posts: 30
Joined: Fri Jul 20, 2007 12:44 am
Location: Chennai

Post by AKUMAR21 »

Simply because using transformer is a costly process eating up a lot of resources and should only be selected for mutlipurpose utility i.e. if you need to do some transformations, merge columns, add some extra columns in the output, reject based on validation failure; all this at the same time then go for the universal stage i.e the transformer. Otherwise if you need to use only one particular functionality then go for other stages. This is why the mutliple functionalities of the transformer has been segregated into various stages in the px. For eg.

1. Simply Map columns from input to output : Copy stage
2.To segeregate records based on certain conditions : Filter stage
3. To concatenate mutliple input columns : Column export stage
4. Split the input column into many columns : Column import stage
......and so on
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Akumar - I just wrote two small test programs to see what the differences are. In a 1-node configuration my transform stage was 20% faster than a column import stage.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

AKUMAR21 wrote:Simply because using transformer is a costly process eating up a lot of resources and should only be selected for mutlipurpose utility i.e. if you need to do some transformations, merge columns, add some extra columns in the output, reject based on validation failure; all this at the same time then go for the universal stage i.e the transformer. Otherwise if you need to use only one particular functionality then go for other stages. This is why the mutliple functionalities of the transformer has been segregated into various stages in the px. For eg.

1. Simply Map columns from input to output : Copy stage
2.To segeregate records based on certain conditions : Filter stage
3. To concatenate mutliple input columns : Column export stage
4. Split the input column into many columns : Column import stage
......and so on
Your information was true prior to version 7.5. In version 7.5 they put a lot of work into making the Transformer stage efficient.
Nowadays, and depending on various factors, the Transformer stage can out-perform the Filter stage in evaluating filtering expressions.
But never believe what you read - test it!
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

AKUMAR21 wrote:Simply because using transformer is a costly process eating up a lot of resources and should only be selected for mutlipurpose utility
As noted, this information is out-of-date with the advent of the 7.5 version of PX.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply