How to split a single row of record in to two row of output

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Prabhakar
Participant
Posts: 10
Joined: Mon Apr 02, 2007 6:28 pm
Location: Melbourne

How to split a single row of record in to two row of output

Post by Prabhakar »

Hi Gurus,

I am having a problem in spliting a single row of records in to 2 output rows

eg:
Input:
cust id Name Lastname age
1 Prab raj 23
2 sam rob 24

output:
Custid Name Lastname
Custid age
1 Prab Raj
1 23
2 Sam rob
2 24


Can any one suggest me which stage to choose to do this function
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

There are several ways of doing it. Use Transformer stage and extend two links. One with "Cust id Name Lastname" and other with "Cust id Name age".
You can use Link Collector stage to merge it or you can directly write it into tow seperate sequential file stage, one with Append option. You can sort based on Cust id if required.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Welcome Aboard :)
You can do a similar thing if your on px job except that you wont have a link collector to collect it. In that case, pass an after job subroutine ExecDOS command to concatenate the two files and sort on id
Something like

Code: Select all

cat file1 file2 | sort > file 3
I supplied unix commands because you are using px on windows which has MKS Toolkit shipped with the product.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

By using Append option in one file will take care of this.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Prabhakar
Participant
Posts: 10
Joined: Mon Apr 02, 2007 6:28 pm
Location: Melbourne

Post by Prabhakar »

but there is no collecter in data Stage what stage to choose to merge the files. sequential file can't have multiple inputs
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

kumar_s wrote:By using Append option in one file will take care of this.
:? Err... two links writing to the same file at the same time, one with the append option? No can do, my friend. If you could manage that in a serial fashion where the overwrite link completes before the append link starts, only then is that an appropriate answer.

You may not be able to have 'multiple inputs' but you can write to two separate files and then combine them after job, or in another job depending on the nature of the 'combining' that needs doing.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

There is a stage called "Link Collector", may be the project that you are using are not customized for that stage. You can do it by Customizing from pallet. But the input for this stage should be of same metadata.
So if you are going to use this, you need to make the file as same metadata on both the link. You may need to concatinate the fields. Or you can use the other approach.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Missed the Craigs point. Even seqential file should have Row level commit. :wink: Should go back to DSGuru2B's option.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
vipshiva
Participant
Posts: 26
Joined: Thu Sep 28, 2006 4:48 am

Post by vipshiva »

Use pivot Stage in Designer.....


Regards,
Siva.
prnath78
Charter Member
Charter Member
Posts: 3
Joined: Wed Aug 24, 2005 11:39 pm

Post by prnath78 »

How about this:

Input ->TX->(link1 with Custid Name Lastname )->| -> funnel(custid)-o/p
->(link2 with Custid age ) ->|

Thanks & Regards,
Ranga
The greatest religion is to be true to your own nature. Have faith in yourselves!. ..swami vivekanandha
michaeld
Premium Member
Premium Member
Posts: 88
Joined: Tue Apr 04, 2006 8:42 am
Location: Toronto, Canada

Post by michaeld »

how about if you use a copy stage and only select the columns that you need for each stream. If you need to filter them later on then use a filter stage.

Transformer stages are kind of slow and should be avoided in high volume jobs if possible.
Mike
us1aslam1us
Charter Member
Charter Member
Posts: 822
Joined: Sat Sep 17, 2005 5:25 pm
Location: USA

Post by us1aslam1us »

prnath78 wrote: Input ->TX->(link1 with Custid Name Lastname )->| -> funnel(custid)-o/p
->(link2 with Custid age ) ->|
Funnel Stage needs same metadata from both streams to work. Dsguru has given clear description, follow that post.
I haven't failed, I've found 10,000 ways that don't work.
Thomas Alva Edison(1847-1931)
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Transformers are not slow, folks. Someone needs to get, in writing, on busting this myth.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
prnath78
Charter Member
Charter Member
Posts: 3
Joined: Wed Aug 24, 2005 11:39 pm

Re: How to split a single row of record in to two row of out

Post by prnath78 »

us1aslam1us wrote:Funnel Stage needs same metadata from both streams to work. Dsguru has given clear description, follow that post.
Sorry.
I agree with you.
The greatest religion is to be true to your own nature. Have faith in yourselves!. ..swami vivekanandha
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

I tried the following scenario.
Two link from Transformer, both to the same sequential file stable, one as Overwrite and other as Append. Interestingly I didn't any error or warnings. And output is as expected!!!
Note : In Server Job.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Post Reply