naming columns from data

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
jasper
Participant
Posts: 111
Joined: Mon May 06, 2002 1:25 am
Location: Belgium

naming columns from data

Post by jasper »

We have an input from an oracle-db which have a column with key-value pairs. There are a lot of changes trough time which pairs are in there.

data examlpe:
Key data
1 name=first;object=table;designer = me
2 name=second;object=script;designer=me

From this data I would like to make records where columnnames are derived from key-names
So:
Key NAME OBJECT DESIGNER
1 first table me
2 second script me

I know this can be done hardcoded with the column-import stage, but since they change a lot I would like something generic (target table can be recreated every run to match columns). Any ideas?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Select from Oracle, convert the delimiter character ";" to a line terminator (x0D) and write to a Sequential File. Then read from the Sequential File. In spite of running sequentially, it's amazingly quick.

Each row read from the sequential file will now have a single name=value pair. The rest is ugly.

Does the output have a finite, or potentially infinite, number of different names? If it's finite, you can conditionally update that column only if the input name matches.

If it's infinite, then you're getting into a realm where DataStage is not designed to go, and no amount of dilithium crystals will help. DataStage works with fixed metadata, not dynamic metadata.

You might be able to get something to work using runtime column propagation, but it's not immediately clear (to me) to see exactly how.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply