Pivoting-unknown number of columns

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
dsnk
Premium Member
Premium Member
Posts: 10
Joined: Fri Jun 15, 2012 12:11 am
Location: USA

Pivoting-unknown number of columns

Post by dsnk »

Hi
I have a requirement to process following:
I/P:
A1|val1;val2;val3.....;valn|C3
B1|val2;var3;var5;..........;varn|C3

O/P:
A1|Val1|C3
A1|val2|C3
...
...
A1|valn|C3
B1|val2|C3
..
..
..
B1|varn|C3

Hear the 2nd column has unknown number of values which has to splitted across for each of the value in column 1


Please help
RPhani
Participant
Posts: 32
Joined: Sun Aug 26, 2012 7:03 am
Location: Hyd

Post by RPhani »

Hi,

Use Transformer Looping concept.

----------------------------------------
R.Phani
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Or use Pivot stage and discard the NULLs subsequently.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
dsnk
Premium Member
Premium Member
Posts: 10
Joined: Fri Jun 15, 2012 12:11 am
Location: USA

Post by dsnk »

Looping is not available in 8.1. I believe this is available from 8.5. Can you please suggest alternatives.
dsnk
Premium Member
Premium Member
Posts: 10
Joined: Fri Jun 15, 2012 12:11 am
Location: USA

Post by dsnk »

The set of values in column-2 is dynamic. Some times it may ave 1 value and at times 50,100, so on.. Is it possible to use pivot for dynamic number of columns ? Please help steps if possible.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The Pivot stage needs a fixed number of columns... as in the maximum you could possiblty get. Then as Ray notes you'd need to filter out the unused values (the NULLS) post-pivot. It works but you can end up creating a ton of records to filter out.

Without transformer looping you may need to do this outside of DataStage, something at the O/S level perhaps.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply