Hi i am creating a job for history processing , and i need to to concatenate all the the non key columns as 1 single column( around 50 ) and at the end i need to split them in their respective values . do we have a ny stage to do.
Thanks
concatenation of columns
Moderators: chulett, rschirm, roy
-
- Charter Member
- Posts: 143
- Joined: Thu Nov 04, 2004 6:53 am
Re: concatenation of columns
jayawant_hsbc wrote:Hi i am creating a job for history processing , and i need to to concatenate all the the non key columns as 1 single column( around 50 ) and at the end i need to split them in their respective values . do we have a ny stage to do.
Thanks
u can use row merger and row splitter
-
- Participant
- Posts: 232
- Joined: Sat May 07, 2005 2:49 pm
- Location: USA
I have used this approach in server jobs, I am not sure whether this works for a parallel job, but you can give it a try.
Without using any special stage, you can do it in a regular sequential file
1) On the input to the sequential file, you will have all your individual columns
2) On the output of the sequential file, you will have all your individual key columns and just one column to hold all your other non key columns
3) Apply whatever transformations that you need and do the reverse in your final sequential file and split the combined column into individual columns
Example :-
Col1 is the key and Col2,Col3 are non-key columns
INPUT FILE ---> (Col1,Col2,Col2) SEQ1 ----> (Col1,COMBO)SEQ2
(Col1,COMBO)SEQ4 ---> (Col1,Col2,Col3)SEQ5 ---> OUTPUT FILE
Without using any special stage, you can do it in a regular sequential file
1) On the input to the sequential file, you will have all your individual columns
2) On the output of the sequential file, you will have all your individual key columns and just one column to hold all your other non key columns
3) Apply whatever transformations that you need and do the reverse in your final sequential file and split the combined column into individual columns
Example :-
Col1 is the key and Col2,Col3 are non-key columns
INPUT FILE ---> (Col1,Col2,Col2) SEQ1 ----> (Col1,COMBO)SEQ2
(Col1,COMBO)SEQ4 ---> (Col1,Col2,Col3)SEQ5 ---> OUTPUT FILE
-
- Participant
- Posts: 7
- Joined: Wed Jan 05, 2005 11:05 pm
Re: concatenation of columns
vcannadevula wrote:jayawant_hsbc wrote:Hi i am creating a job for history processing , and i need to to concatenate all the the non key columns as 1 single column( around 50 ) and at the end i need to split them in their respective values . do we have a ny stage to do.
Thanks
u can use row merger and row splitter
both these stages are in server edition i am working in parallel edition
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
There is the column export stage which can combine multiple columns into a single column. You can pass through the key columns to keep them out of the concatenation.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Re: concatenation of columns
Hi Sandeep,
There is a Column Export and a Column Import Stage in the Parallel Edition. You should be able to do what you want, with these stages. The help files for these stages should assist you well, for this purpose.
Regards,
The Bird.
There is a Column Export and a Column Import Stage in the Parallel Edition. You should be able to do what you want, with these stages. The help files for these stages should assist you well, for this purpose.
Regards,
The Bird.
jayawant_hsbc wrote:Hi i am creating a job for history processing , and i need to to concatenate all the the non key columns as 1 single column( around 50 ) and at the end i need to split them in their respective values . do we have a ny stage to do.
Thanks
Export stage
Hi jayawant,
In my last project we used the Export stage a lot. keep in mind two things:
1. you'll have to keep a matching Schema file for each file you export.
2. we learned that this stage can be pretty heavy.
In order to save time, if we needed to get information from these files after being concatenated, we used the [] option (so we can avoid import stage).
good luck,
Keren
In my last project we used the Export stage a lot. keep in mind two things:
1. you'll have to keep a matching Schema file for each file you export.
2. we learned that this stage can be pretty heavy.
In order to save time, if we needed to get information from these files after being concatenated, we used the [] option (so we can avoid import stage).
good luck,
Keren