You can challenge any decision when you're in the right.
That fact that you challenge a foolish decision doesn't mean that the fools who made it will change their minds, but even getting them to think about it is a start.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Smita, the indicated post shows how to use stage variables. You must sort the data by the grouping keys and partition by them so that they stay together on the same node. The only issue is the last group won't output without some help. The "trigger" event on the stage variables is to only output the final concatenated row when the group changes, but the last group has no next group to indicate a group change. You'll have to add a row to the input for each node so that each node will output its final group. The stage variable recommendation is not the easiest or cleanest solution.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Declare stage variables as follows: StageVar=DSLink3.No StageVar2=StageVar<> StageVar1 StageVar1=StageVar StageVar5=DSLink3.Type:',':DSLink3.Name1:',': DSLink3.Name2 StageVar4=IF StageVar2=1 then DSLink3.No:',': StageVar5 else Trim(StageVar4):',':Trim(StageVar5)
Output has two column with the below derivation: No=DSLink3.No Output=StageVar4
Check for the field length of the Output Column to be suficient to hold the concatenated data
Remove Duplicate Stage:
Pass the rows through a Remove Duplicate Stage and give the key as NO and select Last in the property Duplicate to Retain
You have made my day. Excellent technique, but please consider the partitioning required on a multi-node cluster to keep grouped rows together. Otherwise, the "last" aggregate solution doesn't require the final row output fix. Excellent.
The same solution would work with Server by changing the Remove Duplicates stage with an Aggregator stage using the "last" action.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
smitageorge wrote:
My exact requirement is like this:
source:
NO TYPE NAME1 NAME2
1 A AA W
1 B BB
1 C CC Y
1 D DD Z
Create Temp with metadata like this
NO Type1 Type2 Type3 Type4 Type5 Type6 Type7 Type8 Type9 ....
1 A AA W B BB C CC Y ....
Thanks
smita
Yes, create it like as you have mentioned. Then from the Temp file pass it to a Transformer where if you want to concatenate the Type1 with Type2 with Type 3 and so on.
And there you have it ready for inserting to your target.
Sunshines method is a possible approach too, but I think this is much more simpler if you are not comfortable with staging variables. If you have a very high amount of data then staging variables might hit performance issues.
Let me know how it goes.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Declare stage variables as follows: StageVar=DSLink3.No StageVar1=StageVar<> StageVar1 StageVar2=StageVar StageVar3=DSLink3.Type:',':DSLink3.Name1:',': DSLink3.Name2 StageVar4=IF StageVar2=1 then DSLink3.No:',': StageVar5 else Trim(StageVar4):',':Trim(StageVar5)
Output has two column with the below derivation: No=DSLink3.No Output=StageVar4
Check for the field length of the Output Column to be suficient to hold the concatenated data
Remove Duplicate Stage:
Pass the rows through a Remove Duplicate Stage and give the key as NO and select Last in the property Duplicate to Retain
Below is my source, and I need to concatenate col03 based on Col01 and Col02.
But I couldnt see any Col2 in your example.
If it is simple concatination, you can acheive using transformer.
Sort baseded on Col1. Use stage variable to check for repeated value in Col1, and based on that, concatinate the Col3 with previuos value.
Something like this.