How to split record

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
bollinenik
Participant
Posts: 111
Joined: Thu Jun 01, 2006 5:12 am
Location: Detroit

How to split record

Post by bollinenik »

Hi,
My requiremnt is , I have 1 record in source file and I need to write that to target as 2 records , that means that single record needs to be split into 2 records

Example:
Input is
******* Single record
Col1 col2 col3 col4
a 1 3 4

Output:
****** 2 records
a 1 3
a 4

Like this, Pls help me how to implement in data stage.

Thanks in Advance.
kOTI,
1586-738-7785
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

YOu can do it a couple of different ways.
One way would be to read the file twice using two different sequential file stages, stick a transformer after them. Choose Col1, Col2 and Col3 for the first stream and Col1 and Col4 from the second stream. You can then write it a sequential file (1 from each stream) and in after job subroutine do a cat, or stick a funnel stage and gather the records. But with the funnel stage you will have to specify a dummy column for your second stream so as to make the meta data similar.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
bollinenik
Participant
Posts: 111
Joined: Thu Jun 01, 2006 5:12 am
Location: Detroit

Post by bollinenik »

Just sample I given 4 columns like that i had so many, so useing more sequential file stages is problem here, Can u tell me what is another way,
The total thing is Just we need to split record into multiple records to target.

Thanks in Advance
kOTI,
1586-738-7785
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Well then give us the complete requirement. Looking at your sample data, it tells me two things.
1)Split a record into two, and
2)First three columns in one record and first and last column in second record.
Now your telling me there are going to be multiple records :?
Please provide a full description of your requirements.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
bollinenik
Participant
Posts: 111
Joined: Thu Jun 01, 2006 5:12 am
Location: Detroit

Post by bollinenik »

Thanks Guru,

In the same manner we may have more then 10 input fields and like that we need to split single record into 10 output records, The main thing we need to split record, that's all....Can any body tell me how to implement this splitting record scenario in data stage

Thanks in Advance
kOTI,
1586-738-7785
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

So basically the number of splits should be dynamic. What column in the source record will tell how many records need to be created?
Can you please give two proper examples with data. How the incoming record will look like and on what basis it will be split and on what basis to decide what column goes into what record.
Need requirements man. If I am not able to provide the correct answer, others who can, need this info.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
novneet
Participant
Posts: 28
Joined: Tue Jan 17, 2006 2:19 pm
Location: PUNE(INDIA)

Post by novneet »

For splitting one record into multiple record, the easiest way I can think of will be put a copy stage after reading the data, from copy stage drag the fields you want in the first record link and drag the fields for the second record link and so on for N number of records links.
After this for merging them, u need to put a funnel stage which will merge them.
The only point that you need to keep in mind that each record link coming out of the copy stage which is going to be input for the funnel should have similar meta-data.
Regards,
Novneet Jain
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

novneet, your solution is impressive if the maximum number of splits is known. From what I understand, it needs to be dynamic.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

This looks suspiciously like a job for the Pivot stage to me.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply