Page 1 of 1

How to implement pipeline,partition parallelism &departi

Posted: Tue Aug 03, 2004 10:05 am
by babbu9
Hi
I am new to parallel environment. I am interested in knowing how to implement pipeline, partition and departitioning using parallel extender.
Please suggest

Posted: Tue Aug 03, 2004 10:52 pm
by mandyli
This partition topic already discussed many times in the forum. Please search in the forum or go through the parjdev.pdf document for parallel Processing.

Posted: Tue Aug 03, 2004 11:18 pm
by ray.wurlod
In PX you don't have to. That's the whole point!

Partitioning (both of processing and of data) is the only thing over which you have control, first through the configuration file, which specifies the number of processing nodes to be allocated to the job, and second through the partitioning options on links in your design.

See Parallel Job Developer's Guide, particularly Chapter 1, for a detailed answer to exactly your question.

Re: How to implement pipeline,partition parallelism &dep

Posted: Wed Aug 04, 2004 11:13 pm
by dsdesigner
I guess when babbu9 mentions departioning he means repartioning. I guess this can be achieved by limiting the execution of a data set to fewer nodes. These nodes can be specified by setting the "Node pool and resource constraints" option in the
(data set stage)--->output---->advanced tab of hte data set stagea. Pls correct me if I am wrong.

Posted: Thu Aug 05, 2004 4:51 am
by ray.wurlod
Departitioning might also be the "fan in" needed by some stages, such as the Sequential File stage, where only one writer is permitted. This is still configured - and can be made visible - as a link property.