DS Engine instance "ade" may be in an Inconsistent

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
nelab28
Premium Member
Premium Member
Posts: 28
Joined: Fri Sep 24, 2004 1:25 am

DS Engine instance "ade" may be in an Inconsistent

Post by nelab28 »

Hi

The following error occurs while trying to shutdown DS.

Unable to remove the following shared memory
segment(s) during shutdown:
m 106000 0xadec7511 --rw-rw-rw- root
dstage 17986 8537
m 166002 0xadee7511 --rw-rw-rw- root
dstage 17987 8537
Stopping JobMonApp
JobMonApp has been shut down.
2 error(s) encountered during shutdown procedure.
DataStage Engine 7.5.1.1 instance "ade" may be in an
inconsistent state.


The reason for trying to shut down the DS was :

There was an error message that says "Locked by the administrator, Contact Administrator"(though exactly not the same message).

By logging to the administrator, and executing DS.TOOLS on the project the options for
1. List all the processes
2. List all Locks
do not output any results.


A possible solution now considered is to restart the Unix server itself. This would be done after clearing the segments by logging under root in Unix.
Would it be a good practice to restart DS and also Unix periodically, rather than wait for some error to be hit. Defunct processes are presently being checked and killed manually.
Is there a Parallel job equivalent of "killing deadlock deamon process on server jobs"

Any help is greatly appreciated.

Thanks.
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Hi,
in general I see no reason why the deadlock daemon shouldn't work for both server and parallel jobs.

consult your sys admins for the lock reason.

could you write more about the boot procedure of your server (DS point of view) and how exactly did you try to shut DS down? - check with your sys admins if the users involved might have caused the issue you had.

from my expiriance this is a fairly rare situation you got to, at least for a production machine.
if this is your dev machine, well things hppen when you develop and change things.

AFAIK there shouldn't be a general need to bounce the machine every X time so things will stay in tact, but if you have the privilage of down time now and then, you might decide to do it.

I think you are better off trying to analize what happend and try to avoid things like this.

IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

nelab28,

the shared memory segments are still in use during DS's attempt to shut down; which is why you are getting these warnings.

if you do a"ps -ef|grep ds" you shouldn't get any processes (apart from you own) showing up.

if you execute a "ipcs -mp | grep ade" which processes show up? They should no longer be attached during a shutdown.

Sometimes (I recall that HP platforms were particularly error prone) the sockets would not be released which would cause a restart of DS to fail. The sockets should timeout, though; so a reboot of the server is not usually necessary.

If you post what the "ipcs" command output is to this thread someone might be able to assist.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

If you've shut down DataStage with these types of error and ipcs shows that there are shared memory segment keys of the form 0xadexxxxx, then you can - as root - remove these shared memory segments using the ipcrm command.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply