dsapi processes for a job

MukundShastri · Post by **MukundShastri** » Tue Jul 27, 2004 3:05 am

Can anaybody tell about-
1] what are dsapi processes
2] how many dsapi processes are create on the operating system
3] what is the diffence between dsapi and phantom processes for a datastage job.

Thanks
Mukund

JasonGibson · Post by **JasonGibson** » Tue Jul 27, 2004 4:39 am

dsapi processes are called from the api in DataStage. The calls are initiated in a couple of ways.

1. The datastage job will initiate dsapi slaves based on the number of transforms occuring in the job itself.

2. dsapi calls are also initiated when a call such as dsloginfo is made.

Hope this helps

ray.wurlod · Post by **ray.wurlod** » Tue Jul 27, 2004 2:42 pm

You can probably work out that dsapi stands for "DataStage API".

When a DataStage client connects, it does so through the dsrpc mechanism, requesting the dscs service (unirpc and uvcs for earlier DataStage releases that were built on UniVerse).

After verifying that the user's machine is authorised to request the dscs service, by looking in the dsrpcservices file, the listener process (dsrpcd) initiates the executable associated with this service. The name of the executable associated with dscs is dsapi_server. This process maintains the connection with the client (looking after inactivity timeout for example) and handles requests for activity (retrieval or storage and so on).

A dsapi_server does not do much actual processing. Most of the processing that the client requests is handled by forking a helper process, which runs as dsapi_slave. It is unusual that a dsapi_server process has more than one dsapi_slave at any one time, because the helper processes can terminate once they've finished the task they were given.

So, what you should be seeing are:

one dsapi_server process for each connected DataStage client

zero (or slightly more than zero) dsapi_slave processes per dsapi_server processes

There will also be uvsh processes associated with running jobs. These are outside the scope of the current discussion, which is about dsapi.

kduke · Post by **kduke** » Tue Jul 27, 2004 5:56 pm

Another Full Wurlod. Good answer.

ray.wurlod · Post by **ray.wurlod** » Tue Jul 27, 2004 9:17 pm

Actually I probably should have continued to talk about zombie dsapi processes too, but didn't. The most likely causes are disconnections from parent processes, whether network dropouts, aborted jobs being run in debugger, or whatever.

chulett · Post by **chulett** » Tue Jul 27, 2004 9:37 pm

ray.wurlod wrote:Actually I probably should have continued to talk about zombie dsapi processes too, but didn't.

Too bad, could have scored our first Double Full Wurlod.

Hey, it's not too late to go back and edit your post!

MukundShastri · Post by **MukundShastri** » Wed Jul 28, 2004 3:59 am

Hi Ray,
Thanks for the reply.
As per you material above dsapi_slave processes are caused by Client. We see aound 1600 dsapi_slave processes .But around 5 people accessing the server from client . Then how come these many processes get formed ?

Should I consider these dsapi_slave processes as zombie dsapi processes.

kduke · Post by **kduke** » Wed Jul 28, 2004 5:18 am

Ray

He set you up for a double. Go for it.

ray.wurlod · Post by **ray.wurlod** » Wed Jul 28, 2004 5:54 am

I'll be in that part of the world next week (India, that is, not necessarily the land of 1600 dsapi_slave processes!).

Mukund, can you take a look at the output of this command:

Code: Select all

ps -ef | head -1 ; ps -ef | grep dsapi | grep -v grep

to determine whether they've been around for a long time, or are fairly recent?

And can you tell us whether the deadlock daemon (dsdlockd) is running?

Code: Select all

ps -ef | grep dsdlockd | grep -v grep

This process has, as one of its tasks, the job of cleaning up after defunct processes.

MukundShastri · Post by **MukundShastri** » Wed Jul 28, 2004 7:21 am

Hi Ray,
I am currently in UK.

Today we have less dsapi running. We got 1600 dsapi running day before yesterday and as a immediate fix the server was rebooted.
As of now , I am getting following output from the first command:
UID PID PPID C STIME TTY TIME CMD
dsadm 17629 17619 0 14:17:56 ? 0:00 dsapi_slave 10 9 0
dsadm 17297 17292 1 14:17:54 ? 0:02 dsapi_slave 10 9 0
dsadm 17508 17498 0 14:17:55 ? 0:00 dsapi_slave 10 9 0
dsadm 29704 29679 0 10:34:38 ? 1:33 dsapi_slave 10 9 0
dsadm 28936 28916 0 12:13:41 ? 0:15 dsapi_slave 10 9 0
dsadm 17269 17263 0 14:17:54 ? 0:01 dsapi_slave 10 9 0
dsadm 12188 12159 0 12:46:21 ? 1:49 dsapi_slave 10 9 0
dsadm 17591 17582 0 14:17:56 ? 0:00 dsapi_slave 10 9 0
dsadm 17548 17538 0 14:17:56 ? 0:00 dsapi_slave 10 9 0

And deadlock deamon is not active as the second command gives now output. How do I start the deadlock deomon.

ray.wurlod · Post by **ray.wurlod** » Wed Jul 28, 2004 2:47 pm

The best way is to configure it for auto-start in the dsdlockd.config file in the DataStage engine directory on the server.
Log in as superuser, cd to the DataStage engine directory, invoke UniVerse, and bring up DEADLOCK.MENU; there is an option on this menu for starting the dsdlockd process.
Check that it starts successfully, using the command I gave you earlier.

Knew you'd left Reliance; just assumed you'd stayed in Mumbai!

MukundShastri · Post by **MukundShastri** » Fri Aug 06, 2004 9:23 am

ray.wurlod wrote:The best way is to configure it for auto-start in the dsdlockd.config file in the DataStage engine directory on the server.
Log in as superuser, cd to the DataStage engine directory, invoke UniVerse, and bring up DEADLOCK.MENU; there is an option on this menu for starting the dsdlockd process.
Check that it starts successfully, using the command I gave you earlier.

Knew you'd left Reliance; just assumed you'd stayed in Mumbai!

Superuser - you mean to say I have to logon as 'root' ?

Thanks
Mukund

chulett · Post by **chulett** » Fri Aug 06, 2004 9:59 am

He doesnt' mean superuser as in root... he means the 'admin' user as in 'dsadm' or whatever name you've assigned to that role.

ray.wurlod · Post by **ray.wurlod** » Fri Aug 06, 2004 10:35 pm

He does indeed mean log in as root.

Mukund is running version 5.x.

chulett · Post by **chulett** » Sat Aug 07, 2004 7:28 am

Bummer... sorry.

I started with 3.1 but some of these subtle "do it this way in x.x but this other way in y.y and don't forget it was broken for awhile in x.yr3" have long since fled my mind. I need to get me a copy of your magic notebook, this is all too much for a mere Muggle.