max how many stages we can use in single parallel job

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
venkateshrupineni
Participant
Posts: 15
Joined: Wed May 02, 2012 3:38 am

max how many stages we can use in single parallel job

Post by venkateshrupineni »

max how many stages we can use in single parallel job
dr.murthy
Participant
Posts: 224
Joined: Sun Dec 07, 2008 8:47 am
Location: delhi

Post by dr.murthy »

There is No limitation
D.N .MURTHY
eph
Premium Member
Premium Member
Posts: 110
Joined: Mon Oct 18, 2010 10:25 am

Post by eph »

Hi,

According to IBM, it is a best practice to keep that number under 30, for readability, easy debug and server load.

You can use more of course.

Eric
123gopal
Participant
Posts: 16
Joined: Sat Feb 09, 2013 11:41 am
Location: chennai

Re: max how many stages we can use in single parallel job

Post by 123gopal »

1.Minimize the usage of Transformer(instead of use copy,modify,filter,row generator)
2.Reduce the number of lookups in job design use not more than 20 stages in a job
3.There is no limit for no of stages like 20 or 30 but we can break the job into small jobs then we use dataset stages to store data

etc.....
mgplk
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Re: max how many stages we can use in single parallel job

Post by ray.wurlod »

123gopal wrote:1.Minimize the usage of Transformer(instead of use copy,modify,filter,row generator)
2.Reduce the number of lookups in job design use not more than 20 stages in a job
3.There is no limit for no of stages like 20 or 30 but we can break the job into small jobs then we use dataset stages to store data

etc.....
The first of these has not been true since version 8.1.1.
What is the provenance of the other two?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply