Does pipeline parallism exist in server jobs?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Does pipeline parallism exist in server jobs?

Post by zulfi123786 »

Does pipeline parallism exist in server jobs?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Yes.
-craig

"You can never have too many knives" -- Logan Nine Fingers
bobyon
Premium Member
Premium Member
Posts: 200
Joined: Tue Mar 02, 2004 10:25 am
Location: Salisbury, NC

Post by bobyon »

Dang, Finally a question I could have answered and you beat me to it. :-)
Bob
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You could always go ahead and answer the inevitable next question. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Actually - I am going to ask the inevitable question... "how?"

I was aware that parititioned parallelism could be implemented manually using link partitioners and link collectors. However I thought that server processed the data through the stages one step at a time - preventing pipeline parallelism.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
ddevdutt
Participant
Posts: 47
Joined: Wed Aug 22, 2007 2:38 pm

Post by ddevdutt »

You can achieve pipeline parallelism by using the IPC stage or by turning on inter process row buffering.

asorrell wrote:Actually - I am going to ask the inevitable question... "how?"

I was aware that parititioned parallelism could be implemented manually using link partitioners and link collectors. However I thought that server processed the data through the stages one step at a time - preventing pipeline parallelism.
DD

Success is right around the corner
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

ddevdutt wrote:You can achieve pipeline parallelism by using the IPC stage or by turning on inter process row buffering.
We have a winner! :D
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply