IPC STAGE and Row Buffering

[santoshkumar]

I have a few doubts about how Row Buffering and the IPC Stage works.
As i understand, the IPC stage can be used between two active stages, so that each stage will be executed as seperate process. However, how is the IPC stage related to the option of Inter Process Row Buffering.

Can someone please clarify how the following scenarios work


1) IPC stage introduced between two active stages with Row Buffering disabled

2) IPC stage introduced between two active stages with In Process Row Buffering option checked.

3) IPC stage introduced between two active stages with Inter Process Row Buffering option checked.

4) No IPC stage between two active stages and the Inter Process Row Buffering option checked.


Thanks in advance!

[ray.wurlod]

1) Simply does not work. IPC stage requires row buffering enabled.
2) The two active stages run in the same process, but rows are buffered so that the upstream stage can begin processing a row as soon as it has finished with its predecessor (rather than waiting for the row to pass right through the job).
3) The two active stages run in separate processes; data are passed from one to the other via row buffers in shared memory, owned/controlled by the IPC stage.
4) Same answer as for (2).
What is accomplished is called "pipeline parallelism". For more information read Chapter 2 of the Parallel Job Developer's Guide.