DSXchange

Hi i am creating a job for history processing , and i need to to concatenate all the the non key columns as 1 single column( around 50 ) and at the end i need to split them in their respective values . do we have a ny stage to do.
Thanks

jayawant_hsbc wrote:Hi i am creating a job for history processing , and i need to to concatenate all the the non key columns as 1 single column( around 50 ) and at the end i need to split them in their respective values . do we have a ny stage to do.
Thanks

u can use row merger and row splitter

I have used this approach in server jobs, I am not sure whether this works for a parallel job, but you can give it a try.

Without using any special stage, you can do it in a regular sequential file

1) On the input to the sequential file, you will have all your individual columns

2) On the output of the sequential file, you will have all your individual key columns and just one column to hold all your other non key columns

3) Apply whatever transformations that you need and do the reverse in your final sequential file and split the combined column into individual columns

Example :-
Col1 is the key and Col2,Col3 are non-key columns

INPUT FILE ---> (Col1,Col2,Col2) SEQ1 ----> (Col1,COMBO)SEQ2


(Col1,COMBO)SEQ4 ---> (Col1,Col2,Col3)SEQ5 ---> OUTPUT FILE

vcannadevula wrote:

jayawant_hsbc wrote:Hi i am creating a job for history processing , and i need to to concatenate all the the non key columns as 1 single column( around 50 ) and at the end i need to split them in their respective values . do we have a ny stage to do.
Thanks

u can use row merger and row splitter

both these stages are in server edition i am working in parallel edition

There is the column export stage which can combine multiple columns into a single column. You can pass through the key columns to keep them out of the concatenation.

Hi Sandeep,

There is a Column Export and a Column Import Stage in the Parallel Edition. You should be able to do what you want, with these stages. The help files for these stages should assist you well, for this purpose.

Regards,

The Bird.

jayawant_hsbc wrote:Hi i am creating a job for history processing , and i need to to concatenate all the the non key columns as 1 single column( around 50 ) and at the end i need to split them in their respective values . do we have a ny stage to do.
Thanks

Hi jayawant,

In my last project we used the Export stage a lot. keep in mind two things:
1. you'll have to keep a matching Schema file for each file you export.
2. we learned that this stage can be pretty heavy.

In order to save time, if we needed to get information from these files after being concatenated, we used the [] option (so we can avoid import stage).

good luck,
Keren