IPC Buffer Size not exceeding 1024Kb

[asitagrawal]

Hi,
I have enabled Row -Buffering, inter-process, in my jobs.
In the Job prperties (Performance tab) I am unable to set
Buffer Siez > 1024 Kb.
If I type , say 4096, it changes back to 1024 as I tab out of it.
Anyways, I am using a Job Parameter also to set the IPC buffer size at individual IPCs.

Which one will be ffective, the one at individual IPCs or the Job level?

Also this 'Kb' 0 Kilo Bits or Kilo Bytes.
Acco to me Kb = Kile Bits and KB = Kile Bytes.


Thx for ur answers.

[ArndW]

The lower level definition will take precedence. But setting a buffer to such a high value will have no benefit to job speed at all, in fact it might even cause the job to run slower.

to quote myself from some thread a long time ago:

...you might not be using explicit IPC stages, but by specifying inter-process buffering you are effectively making all of your active stages into separate processes that communicate with the other stages via named pipes - so you are ending up doing IPC.

A pipe between processes can be visualized something like a bathtub. The writing process fills the tub with water, the reading process is the drain and empties the bathtub. Your buffer size is the size of the bathtub. If the filler & emptier are working at about the same speed, then the size of the bathtub really doesn't matter. The buffering effect comes when one or the other only works in bursts; then you want the tub to be full enough to let the draining continue when filling stops or that there is enough room to fill up for a while if the draining stops.

The timeout is just used as an abort. If the tub is empty/full and the waiting process has to wait for longer than the the timeout interval an error is raised - the assumption is that the something has gone wrong with one of the processes and the job is made to abort.

So usually whether your buffer is sink-sized, bathtub-sized or swimming-pool sized makes no difference to the actual processing rates, but allocating large buffers will waste memory.

[ray.wurlod]

The behaviour of the control suggests that the upper limit is 1024KB. In a quick survey I can not find a limit documented.

[asitagrawal]

Thanks Ray & ArndW!

Any comments on this..

Also this 'Kb' 0 Kilo Bits or Kilo Bytes?
(Acco to me Kb = Kile Bits and KB = Kile Bytes.)
What do u say?

Thx

[ray.wurlod]

In DataStage, KB is always 1,024 bytes, MB is always 1,024,576 bytes, and GB is 1,073,741,824 bytes. Never bits.

Or 1,024; 10,24,576 and 1,07,37,41,824 on the sub-continent!