split

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
new_ds_man
Participant
Posts: 18
Joined: Tue Jul 08, 2008 1:12 pm

split

Post by new_ds_man »

Hi All,

I have a requirement to split the incoming data based on a key column

data comes in as:
Key column column2 column3 column4
A 1 2 3
A 4 5 6
A 19 20 21
B 7 8 9
B 10 11 12
C 13 14 15
C 16 17 18
and so on....................

I need all As,Bs,Cs in separate O/P links.
Help appreciated.

Thanks in Advance.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

For a finite domain use a Transformer, Switch or Filter stage to effect the split.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
puliram
Participant
Posts: 20
Joined: Mon Apr 28, 2008 6:10 pm

Re: split

Post by puliram »

new_ds_man wrote:Hi All,

I have a requirement to split the incoming data based on a key column

data comes in as:
Key column column2 column3 column4
A 1 2 3
A 4 5 6
A 19 20 21
B 7 8 9
B 10 11 12
C 13 14 15
C 16 17 18
and so on....................

I need all As,Bs,Cs in separate O/P links.
Help appreciated.

Thanks in Advance.

------------------------------------------------------------------
Use transformer, Place two output links
in one link map colmn 1
and in other map column2, column3 and column4
Kryt0n
Participant
Posts: 584
Joined: Wed Jun 22, 2005 7:28 pm

Post by Kryt0n »

Assuming you never know the key values, in a sequence job, run an awk script to get the distinct list of keys then loop through a job providing the key as input with a transformer or filter stage working off the input key.

Suitability really depends on size of file/number of distinct keys as it won't be quick.

Alternatively, a shell script to do it...
Post Reply