Page 1 of 1

How to limit the number of output files to a COPY stage

Posted: Sun May 25, 2008 4:15 pm
by dsean
Hi,
I am using the Enterprise edition for the first time and I want to know how to send dynamically the number of output files that we want to create or copy using the COPY stage in Enterprise edition. I did a search for this but did not find anything about this. Any help can be greatly appreciated.


Thanks in advance,
Sean

Posted: Sun May 25, 2008 7:49 pm
by ray.wurlod
Copy stage does not create output files. The Copy stage creates as many copies of its input stream as there are output links from the stage. This is done in memory - no files are involved.

Posted: Sun May 25, 2008 7:57 pm
by dsean
Ray,
Thanks for your reply. may be the question that I posted is not clear.

Lets say I have a Copy stage with one input link and 4 output links . Now without changing the job design , Can I limit the number of output copies to 3 during runtime from the COPY stage??

Once again, I appreciate your help.

Thanks in advance,
Sean

Posted: Sun May 25, 2008 8:41 pm
by chulett
Not that I am aware of.

Posted: Mon May 26, 2008 12:17 am
by ray.wurlod
More clearly, No.

You can do achieve what you describe using a Filter stage or a Transformer stage. You may be able to do it with a Switch stage (if the condition for output link selection is based on values in the data).

If a Copy stage has N output links it will always generate N streams of output (for N >= 0).

Posted: Mon May 26, 2008 7:01 am
by chulett
Thanks for the more clearly... was pretty sure but not 100% Wurlod proof sure. :wink: