number of jobs in a project

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kirankota79
Premium Member
Premium Member
Posts: 315
Joined: Tue Oct 31, 2006 3:38 pm

number of jobs in a project

Post by kirankota79 »

is there any limit on the number of jobs to be created in a single project
nick.bond
Charter Member
Charter Member
Posts: 230
Joined: Thu Jan 15, 2004 12:00 pm
Location: London

Post by nick.bond »

I don't think so, at least can't think of a reason for there being a limit, but someone else will need to confirm that. As long as you organize your jobs well you should be able to have a large number of them before it becomes unwieldy. We currently have just under 1000 jobs and have no issues.

It would be interesting to know what's the largest number anyone has worked with in one project.......just because..... :?:
Regards,

Nick.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

There is a limit on some operating systems; it's actually the number of subdirectories in a directory. On Solaris 5.2 that limit is 32K, which effectively limits the maximum number of jobs to around 5000. It's mainly cropped up as a problem in the Version Control project, and the workaround is to use multiple Version Control projects.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
csrazdan
Participant
Posts: 127
Joined: Wed May 12, 2004 6:03 pm
Location: Chicago IL

Post by csrazdan »

ray.wurlod wrote:There is a limit on some operating systems; it's actually the number of subdirectories in a directory. On Solaris 5.2 that limit is 32K, which effectively limits the maximum number of jobs to around ...
Yes there is a limitation. This is because DataStage being a 32 bit application can address only 32K address space.

On Solaris:
When the number of sub-directories under the project directory reaches 32804, the UV command CREATE.FILE will fail and you cannot create any more jobs in this project.

On AIX:
The above statement is true for AIX also but we have a workarround. The file system on which project directory is created should have variable VIX (Variable Inode) set to YES

On Windows:
Sorry... No Experience

Hope it helps...
Assume everything I say or do is positive
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

This has nothing to do with DataStage being a 32bit app but is strictly a limitation imposed by the O/S.
-craig

"You can never have too many knives" -- Logan Nine Fingers
csrazdan
Participant
Posts: 127
Joined: Wed May 12, 2004 6:03 pm
Location: Chicago IL

Post by csrazdan »

chulett wrote:This has nothing to do with DataStage being a 32bit app but is strictly a limitation imposed by the O/S. ...
Yes you are correct.
I remember even touch command failed to create a file.
Assume everything I say or do is positive
Post Reply