Limit to Multiple instance of sequences and D's parallel job

mansoor_nb · Post by **mansoor_nb** » Mon Jun 09, 2014 11:11 pm

Hi,
DS V9.1
AIX V7.0

We have designed a common generic RCP driven DS parallel jobs to read multiple types of source files like EBCDIC & ASCII formats having multiple file layouts using a set of filter, SF, Generic stage to load the data into table in teradata database.

I wanted to know is there any limit on number of instances of the jobs. I suspect there may be issues related to the hardware infrastructure as the DS jobs will be executed in multiple nodes (4 nodes) through DataStahe Sequences which is also a multi instanced.
If some one can put their thoughts on this then it would be really helpful. Thanks for all your support and help.

Thanks & Regards
Mansoor

ArndW · Post by **ArndW** » Tue Jun 10, 2014 6:37 am

There is no limit on the number of parallel instances you can run from the software side, your UNIX box will continue spawning new processes until it slows down from overloaded hardware (context switch, swapping, I/O, CPU, etc.) and then newly spawned processes in jobs will begin getting timeouts and the jobs will abort.

ray.wurlod · Post by **ray.wurlod** » Tue Jun 10, 2014 4:34 pm

Or, since you're on version 9.1, you can configure the workload management queues so that there is a finite upper limit on the number of jobs, the amount of CPU or memory consumed, etc., so that "surplus" jobs don't abort but, instead, wait until resources become available.

qt_ky · Post by **qt_ky** » Tue Jun 10, 2014 7:37 pm

With the default 9.1 settings, approximately 150 jobs can run concurrently and/or be queued (according to IBM).

See the following two IBM Technotes FAQ links for details.

1. Describes settings you may change to tune towards running a larger number of jobs:

Tuning the UVCONFIG parameters for IBM InfoSphere Information Server Workload Management

2. Includes some history of default settings that have evolved across various major releases:

Use of the tunable parameters in the UVCONFIG file

mansoor_nb · Post by **mansoor_nb** » Wed Jun 11, 2014 3:48 am

Hi All,
Thanks for your invaluable response.
Based on the inputs provided and the information available as per the links shared, i have checked the uvconfig file in our PROD box. The value defined for T30FILE & RLTABSZ are 4096 & 480 respectively. This value will let 900 jobs to be executed or queued concurrently

My Next question, based on what proportion these values can be increased in order to get the higher number of jobs (greater than 900 jobs) getting executed or queued concurrently.

Thanks for your help

Thanks & Regrads
Mansoor

chulett · Post by **chulett** » Wed Jun 11, 2014 7:08 am

I'm not sure you should be worrying about that. I very much doubt that your hardware will support 900 concurrent Parallel jobs, let alone any sort of a higher number.