Multiple Inputs to one CSV

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
MrBlack
Participant
Posts: 125
Joined: Wed Aug 08, 2012 8:57 am

Multiple Inputs to one CSV

Post by MrBlack »

How do I use multiple inputs to output to a single CSV? Do I use the Row Merge stage or just the Merge stage, something else? I've been experimenting and I can't get anything to work.

For manageability I have split a dozen database queries that I want to union into a single CSV file.
bj_ds7
Premium Member
Premium Member
Posts: 32
Joined: Fri Dec 13, 2013 2:26 pm

Re: Multiple Inputs to one CSV

Post by bj_ds7 »

You can use funnel stage if all the input files records have to be stacked up in a csv file.(if metadata are same) If its a column wise addition you can use join stage or merge stage or look up stage
Thanks & Regards!
BJ
MrBlack
Participant
Posts: 125
Joined: Wed Aug 08, 2012 8:57 am

Post by MrBlack »

I'm on the server edition, I don't have the funnel stage available to me. I think the Merge Stage might be the right track but I'm confused that it doesn't have any input links, only output?

http://pic.dhe.ibm.com/infocenter/iisin ... tages.html
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

The server Merge stage is used to join sequential files. You're looking for the Link Collector stage for your need.

Mike
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Hi,
If I understood correctly,
you have severl source files that need to eventually be written to a single sequential files with the CSV format.

So there are several solutions available, depending on the similarity or lack of, between your source files.

Writing to a sequential files simultaniously from more then 1 process/link at the same time should not work or will not work correctly -even if you managed to produce such a job.

Scenario 1 - option 1:
* All sopurce files have the same definition
- use filter command to TYPE/cat all the files to a single input streem
TYPE - for windows server ; cat for unix/linux server.

Scenario 2 - option 2:
* Source files have different definition (table definition - beyond column names)
- Read all files seperatly in the same job, use transformer to
get them to the same output file format needed for the CSV target file
- use link collector stage to collect the data using the round robin method
and output the collected data to the target CSV file.

You can also use option 2 in the Scenario 1 example as well.

P.S.
It wouldn't surprise me if there are other solutions available,
theses are 2 from the top of my head.

IHTH (I Hope This Helps),
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
Post Reply