split records

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mail2hfz
Premium Member
Premium Member
Posts: 92
Joined: Thu Nov 16, 2006 8:51 am

split records

Post by mail2hfz »

Hi,

I have a requirement to split the sub records. Input contains following fields.
Col1 ->a1|a2|a3
Col2 ->abc
Col3->10
My output should be:
col1 col2 col3
a1 abc 10
a2 abc 10
a3 abc 10
I achieved this by using a transformer. But my col1 may contain upto 50 delimited values,which resluts in 50 links to transformer, so I was asked to go for an alternative approach. Can anyone suggest me an alternative approach to achieve this.

Thanks
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

In an upstream Transformer stage split a1 into 50 columns and preserve a2 and a3. Downstream of that use a Pivot stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
mail2hfz
Premium Member
Premium Member
Posts: 92
Joined: Thu Nov 16, 2006 8:51 am

Post by mail2hfz »

Appreciate your response. It works but we are not sure of the maximum columns to split. If we know the maximum value then we can split into so many columns and use pivot but we don't know the max value. I should have mentioned this while posting but I apologize for that. Is it a good way to form the incoming record into a vector and then use split sub vector. Will this work in this case?

Thanks
sud
Premium Member
Premium Member
Posts: 366
Joined: Fri Dec 02, 2005 5:00 am
Location: Here I Am

Post by sud »

You have to create a buildop where you have to count the number of delimiters and then spit out as many records as required.
Last edited by sud on Thu Jul 24, 2008 8:30 am, edited 1 time in total.
It took me fifteen years to discover I had no talent for ETL, but I couldn't give it up because by that time I was too famous.
mail2hfz
Premium Member
Premium Member
Posts: 92
Joined: Thu Nov 16, 2006 8:51 am

Post by mail2hfz »

Any other approach besides build ops?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Same approach as I outlined earlier. Split into (MAX) columns, filling with in-band nulls, pivot, then filter out un-needed rows.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply