DSXchange

HI All,

The Datastage server is running slow because of a particular design.

Earlier we had a non-multi instance datastage job calling
a multinstance job. Now we changed the application design to
multi-instance datastage job calling another multi-instance job.

This causes the number of dsapi in the unix box to run upto 500
when only four such jobs are running.

Can anyone tell how to solve this by keeping the multi-instance feature.

Regards
Sreenivasulu

The only solution you need is to do some better design, this is not a problem with single- or multi-instace jobs but at a lower level. At a very abstract level you might say that it takes 5000 seconds do to a job. If you split this up into 500 jobs that each need only 10 seconds you will get a box flooded with processes and it will still take 5000 wall-clock seconds for these jobs to run. On bigger SMP systems you can reduce this since job can truly run in parallel; but you will still have a lot of processes (this is part of what PX does in order to run so quickly).

A multi-instance job calling another multi-instance still is only 2 job processes. So you must be doing quite a bit more to get so many jobs running at the same time, and slowly.

Running more jobs on the same machine without increasing cpus means things will run slower. What's the problem? Are you running any diagnostic tools to show you cpu, memory, and disk utilization?

Processes dsapi_server and dsapi_slave have nothing whatsoever to do with running jobs; rather these are the agent processes for connected DataStage clients.

Depending on version, server jobs run as uvsh or dssh, while processes in parallel jobs run as lots of osh processes - one conductor, one section leader per processing node, and one for each uncombined operator.

You need to find out why the dsapi processes are not being cleaned up. What exact version of DataStage are you running? Do you auto-start the deadlock daemon?

HI All,

Thanks for your suggestions. I will look into these and get back to you.

I am facing this problem in Server Jobs Datastage 5.2.

Sorry for mentioning Server Datastage 7

Regards
Sreenivasulu