Max Number of Job

sangi1981 · Post by **sangi1981** » Fri Oct 30, 2015 4:59 am

Hi all,
We encountered max number (32K) of existing directory on JFS, inside a Datastage Project.

We can't divide Project.

How can we solve the problem?
Changing FS? Changing DS version?

Some help is usefull.

Cheers
Sandro

qt_ky · Post by **qt_ky** » Fri Oct 30, 2015 5:21 am

Yes, there is a hard 32K subdir. limit! Changing DS version will not make any difference here; it is known OS / file system limitation.

As far as I know, you have to create a new (overflow) project and move a set of jobs there, and consider the original project to be full.

Out of curiosity, how many jobs are in your full project? Perhaps in the 5000 to 6000 range?

Other than that, you could google about file system options on AIX or ask your UNIX Admin about such options, if any exist.

Why are you saying you cannot split the project? I'm sure you can.

qt_ky · Post by **qt_ky** » Fri Oct 30, 2015 5:33 am

Check out this technote. It does give another option, but it sounds so much worse than dividing the jobs across projects...

Maximum number of InfoSphere DataStage Jobs in a project

sangi1981 · Post by **sangi1981** » Fri Oct 30, 2015 6:08 am

Thank you for your answer.

We have over 4500 job.....

Our customer, for internal reason, doesn't want to divide project...it is so.

He prefer to apply some other solution, to avoiding splitting project.
Due to fact that changing DS version is indifferent to number of dir in project, the solution is to verify with system administrator what kind of options wwe have.
Is it right?

What other FS exists for AIX?

Thank you

chulett · Post by **chulett** » Fri Oct 30, 2015 7:02 am

sangi1981 wrote:What other FS exists for AIX?

As noted, that's a question for your SA. And I would bet that "... and they are willing to support" gets added to your question.

sangi1981 also wrote:Our customer, for internal reason, doesn't want to divide project...it is so.

Sometimes it's not about what is wanted but what can be done. You made need to make it... not so.

PaulVL · Post by **PaulVL** » Fri Oct 30, 2015 9:33 am

The customer is not always right.

chulett · Post by **chulett** » Fri Oct 30, 2015 11:36 am

So... out of curiousity, how did you resolve this?

sangi1981 · Post by **sangi1981** » Mon Nov 02, 2015 5:48 am

Hi Craig,
our system admin proposed to migrate all to AIX/GPFS, but a check is in progress with support, to verify cost and time.
Bye