I recently saw a statement that it would nice if DataStage had a continuous reading funnel. Does anyone have any idea what that might mean? I couldn't find anything meaningful on Google about this.
Thanks,
Tony
continuous reading funnel
Moderators: chulett, rschirm, roy
-
tonystark622
- Premium Member
- Posts: 483
- Joined: Thu Jun 12, 2003 4:47 pm
- Location: St. Louis, Missouri USA
A funnel is used to converge multiple primary data streams into a single primary data stream.
They're probably referring to the issue with the Server link collector. It does a round-robin read off each incoming link. If there is not a row at an incoming link the collector will stall there waiting for a row. Once it gets a row it moves to the next link and waits for a row. Unless rows are stacked behind every link, the collector stalls on the slowest feeding link.
This is why you're better off either writing each output link to a file or a named pipe, so that output spools at the speed in which it is produced.
Kenneth Bland
They're probably referring to the issue with the Server link collector. It does a round-robin read off each incoming link. If there is not a row at an incoming link the collector will stall there waiting for a row. Once it gets a row it moves to the next link and waits for a row. Unless rows are stacked behind every link, the collector stalls on the slowest feeding link.
This is why you're better off either writing each output link to a file or a named pipe, so that output spools at the speed in which it is produced.
Kenneth Bland
-
tonystark622
- Premium Member
- Posts: 483
- Joined: Thu Jun 12, 2003 4:47 pm
- Location: St. Louis, Missouri USA
Hello --
The funnel stage for Parallel Extender also suffered from the same problem for 6.0, as in waiting in a round-robin style. Fortunately, it have been noted for 7.0 that the Funnel stage's performance problem was fixed. I have not yet tested the validity of that advice.
-T.J.
* * *
... now if this can make breakfast, my life is complete.
The funnel stage for Parallel Extender also suffered from the same problem for 6.0, as in waiting in a round-robin style. Fortunately, it have been noted for 7.0 that the Funnel stage's performance problem was fixed. I have not yet tested the validity of that advice.
-T.J.
* * *
... now if this can make breakfast, my life is complete.