Multiple Rows into one single row, comma seperated.
Moderators: chulett, rschirm, roy
Multiple Rows into one single row, comma seperated.
Hi,
This is my first post here, been reading a lot of posts on this website, pretty interesting.
Anyway, I am facing this issue:
I have the following data
<pre>
DM_OPERATION ID REASON_CODE
---------------------- -------------------
4 158
4 156
4 865
4 901
4 483
</pre>
I need to output them as
<pre>
DM_OPERATION ID REASON_CODE
---------------------- -------------------
4 158,156,865,901,483
</pre>
Read a couple of posts say to use Pivot, but I believe it's wrong. Not to mention that I am unable to configure it correctly.
The job is a bit complex, there are many stages before, and some after the above is done. I need the Operation_ID in order to merge it with other set of data. And I need the data to be comma separated because the customer asked me to!
I'd appreciate any advice. What stage to use, and quick example if possible.
Thanks.
This is my first post here, been reading a lot of posts on this website, pretty interesting.
Anyway, I am facing this issue:
I have the following data
<pre>
DM_OPERATION ID REASON_CODE
---------------------- -------------------
4 158
4 156
4 865
4 901
4 483
</pre>
I need to output them as
<pre>
DM_OPERATION ID REASON_CODE
---------------------- -------------------
4 158,156,865,901,483
</pre>
Read a couple of posts say to use Pivot, but I believe it's wrong. Not to mention that I am unable to configure it correctly.
The job is a bit complex, there are many stages before, and some after the above is done. I need the Operation_ID in order to merge it with other set of data. And I need the data to be comma separated because the customer asked me to!
I'd appreciate any advice. What stage to use, and quick example if possible.
Thanks.
Welcome.
It most certainly is a pivot, actually a "vertical pivot" of rows to columns. However, you are correct in that the actual Pivot stage only supports the other direction - columns to rows or a horizontal pivot - until the upcoming 8.5 release, at least from what I seem to remember I've read. So, in the meantime, you get to do it yourself!
If you've been searching the forums, you'll find most advice on vertical pivoting says to search the forum. Advice is out there, it's just the wheat is buried amongst a ton of chaff.
What comes to mind is transformer stage variable work along with an Aggregator. You need to do basic 'group change detection' so you know when your DM_OPERATION_ID value changes and you can do that all in stage variables or with a preceding Sort stage with a Key Change column added. If you go the latter route, make sure you tell it "Don't sort, already sorted".
In the transformer, accumulate REASON_CODE values in a single field and go ahead and output the single field. The Aggregator comes into play to capture the 'Last' record from each group of DM_OPERATION_ID values. So, output looking something like this:
Use the key change column being 1 or 'true' to know when to reset the accumlating stage variable back to just the initial value. Every subsequent row where the key change column is 0 or 'false' you concatenate a comma and the current value onto the accumulating value.
Hope that helps and can be integrated into your current design. I'm sure there are other ways to skin this cat but that's the typical approach to this.
It most certainly is a pivot, actually a "vertical pivot" of rows to columns. However, you are correct in that the actual Pivot stage only supports the other direction - columns to rows or a horizontal pivot - until the upcoming 8.5 release, at least from what I seem to remember I've read. So, in the meantime, you get to do it yourself!
If you've been searching the forums, you'll find most advice on vertical pivoting says to search the forum. Advice is out there, it's just the wheat is buried amongst a ton of chaff.
What comes to mind is transformer stage variable work along with an Aggregator. You need to do basic 'group change detection' so you know when your DM_OPERATION_ID value changes and you can do that all in stage variables or with a preceding Sort stage with a Key Change column added. If you go the latter route, make sure you tell it "Don't sort, already sorted".
In the transformer, accumulate REASON_CODE values in a single field and go ahead and output the single field. The Aggregator comes into play to capture the 'Last' record from each group of DM_OPERATION_ID values. So, output looking something like this:
Code: Select all
4 158
4 158,156
4 158,156,865
4 158,156,865,901
4 158,156,865,901,483 <- this row is passed by the Aggregator
5 184
5 184,163
-etc-
Hope that helps and can be integrated into your current design. I'm sure there are other ways to skin this cat but that's the typical approach to this.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Use of stage variables
HI,
Sort and Partition the data on first column, use stage variables to store second column. Concat the values of second column into the stage variable for repeatition of first column. (You might want to use a couple of stage variables to get the result).
You will get below result-
4 158
4 158,156
4 158,156,865
4 158,156,865,901
4 158,156,865,901,483
5 184
5 184,163
Then sort and remove duplicates to get this-
4 158,156,865,901,483
5 184,163
Sort and Partition the data on first column, use stage variables to store second column. Concat the values of second column into the stage variable for repeatition of first column. (You might want to use a couple of stage variables to get the result).
You will get below result-
4 158
4 158,156
4 158,156,865
4 158,156,865,901
4 158,156,865,901,483
5 184
5 184,163
Then sort and remove duplicates to get this-
4 158,156,865,901,483
5 184,163
-
- Participant
- Posts: 135
- Joined: Tue Aug 14, 2007 4:27 am
- Location: Mumbai
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 96
- Joined: Fri Aug 20, 2010 8:51 am
marpadga18 - if that's not enough to help, please start your own topic and let us know what problem you are trying to solve and what issues you are having...iHijazi wrote:I must say that ankesh solution worked like a charm, thanks buddy!! Simple and straightforward
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 96
- Joined: Fri Aug 20, 2010 8:51 am