two streams into one??
Moderators: chulett, rschirm, roy
two streams into one??
Hi!!
I have a question. How in DataStage server 7.5 can I make one data stream from few others streams??
For example: in one stream I have 100 rows and in another one 100 rows. Both streams have the same columns but different data. Both streams processing is parralel.
How can I make one stream with the same columns with (in this example) 200 rows??
Regards
I have a question. How in DataStage server 7.5 can I make one data stream from few others streams??
For example: in one stream I have 100 rows and in another one 100 rows. Both streams have the same columns but different data. Both streams processing is parralel.
How can I make one stream with the same columns with (in this example) 200 rows??
Regards
LUK
Luk,
this is what the MERGE stage does, it will merge your two data streams into one.
this is what the MERGE stage does, it will merge your two data streams into one.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Hi,
How about using a filter command that concatenates both files and gives you a single output link? (should be good if the files are not big, better then merge in any case).
using link collector will probably pose issues in this case.
IHTH,
How about using a filter command that concatenates both files and gives you a single output link? (should be good if the files are not big, better then merge in any case).
using link collector will probably pose issues in this case.
IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Time is money but when you don't have money time is all you can afford.
Search before posting:)
Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
It is actually possible to merge two streams without using files as buffers. I always use files to source and collect my data, but a quick read of the documentation tells me you can use inter process row buffering. You can find this option in the performance tab of the properties page to enable active to active links. Try this, and let us know how you get on.
Just for the record, this is a PX answer. The confusion comes from having two stages, performing two different (yet somewhat similiar) functions, with the same name - MERGE.ArndW wrote:Luk,
this is what the MERGE stage does, it will merge your two data streams into one.
The Merge in PX is similar to the Link Collector in Server, while the Merge in Server actually 'merges' two flat files by treating them like tables (behind the scenes) and allowing them to be 'joined' to create a single input source.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Craig,
I have noticed that many people have experienced problems with the link collector stage, when enabling inter process row buffering (ds_ipcopen() - Error in open). Have you experienced issues with this method? My workaround was to not use any row buffering, instead I write data to multiple seq files as source, and output data to a seq file.
Luk, I wouldn't be surprised if you encounter this too!
I have noticed that many people have experienced problems with the link collector stage, when enabling inter process row buffering (ds_ipcopen() - Error in open). Have you experienced issues with this method? My workaround was to not use any row buffering, instead I write data to multiple seq files as source, and output data to a seq file.
Luk, I wouldn't be surprised if you encounter this too!