Page 1 of 1

split records

Posted: Mon Jul 21, 2008 11:10 am
by mail2hfz
Hi,

I have a requirement to split the sub records. Input contains following fields.
Col1 ->a1|a2|a3
Col2 ->abc
Col3->10
My output should be:
col1 col2 col3
a1 abc 10
a2 abc 10
a3 abc 10
I achieved this by using a transformer. But my col1 may contain upto 50 delimited values,which resluts in 50 links to transformer, so I was asked to go for an alternative approach. Can anyone suggest me an alternative approach to achieve this.

Thanks

Posted: Mon Jul 21, 2008 3:14 pm
by ray.wurlod
In an upstream Transformer stage split a1 into 50 columns and preserve a2 and a3. Downstream of that use a Pivot stage.

Posted: Tue Jul 22, 2008 7:32 am
by mail2hfz
Appreciate your response. It works but we are not sure of the maximum columns to split. If we know the maximum value then we can split into so many columns and use pivot but we don't know the max value. I should have mentioned this while posting but I apologize for that. Is it a good way to form the incoming record into a vector and then use split sub vector. Will this work in this case?

Thanks

Posted: Tue Jul 22, 2008 8:47 am
by sud
You have to create a buildop where you have to count the number of delimiters and then spit out as many records as required.

Posted: Tue Jul 22, 2008 12:43 pm
by mail2hfz
Any other approach besides build ops?

Posted: Tue Jul 22, 2008 3:26 pm
by ray.wurlod
Same approach as I outlined earlier. Split into (MAX) columns, filling with in-band nulls, pivot, then filter out un-needed rows.