multi-instance :can't start together

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
telenet_bi
Premium Member
Premium Member
Posts: 33
Joined: Wed Jul 23, 2008 7:33 am
Location: Mechelen, Belgium
Contact:

multi-instance :can't start together

Post by telenet_bi »

Hi,

We are currently migrating from 7.5 to 8. All our jobs indivdually run fine, but when we start scheduling everything fails. We've been able to track this down to a multi-instance job which is used in all our flows. If this is triggered to start too much at the same moment(with other invocationid's ) it fails. (status 99 or 3).

We've created a workflow which is just triggering this job 5 times in parallel. This fails even without any other activities on the server. CPU is then used for about 5% .

IBM support says:
Following my investigations on DataStage architecture it appears that you cannot run more job in a second than the number of CPU installed on the server, there would be a dispatch issue. This is valid for DataStage 8
I find this very hard to believe and if it is the case we have no choice but to return to version 7 where we can trigger a lot more jobs.

Anybody had simular problems and what would then be the solution?
telenet_bi
Premium Member
Premium Member
Posts: 33
Joined: Wed Jul 23, 2008 7:33 am
Location: Mechelen, Belgium
Contact:

Post by telenet_bi »

after some searching here, I feel there is a link with the problem in viewtopic.php?t=115767&highlight=

I'm going to disable the purging to make sure and will update later.
telenet_bi
Premium Member
Premium Member
Posts: 33
Joined: Wed Jul 23, 2008 7:33 am
Location: Mechelen, Belgium
Contact:

Post by telenet_bi »

problem was indeed multiple instance jobs in combination with autopurge of logs after 10 runs.
disabling autopurge, or setting it to 5days seems to work.
Post Reply