Jobs are in Running status

arunpramanik · Post by **arunpramanik** » Wed Aug 18, 2010 3:05 am

Recently I had face a issue. I found the jobs are in running status while it should be Aborted.

Here I am talking about a DataStage server (hosted on a virtual machine) retrieves data from MS SQL Server hosted a different Cluster server

I have went through the earlier post viewtopic.php?t=130554

We found that the database server was patched at the time concerned; and MS SQL server cluster did fail over. DBA informed that the already running job was killed and they were not recovered.

Now my question is
1. if the MS SQL server jobs / process were killed, how come Datastage jobs are still in running state?

2. Even if the DB connectivity fails or DB is shut down while jobs are running, I think jobs get aborted. Isn't it?

3. What happens if DB Cluster fails over during the jobs are running based on the current installation scenario?

Requesting to help me understand why the jobs were still running in this scenario

-- Arun

ArndW · Post by **ArndW** » Wed Aug 18, 2010 3:27 am

Was DataStage reading from the database when it was failed over? If DataStage doesn't get an error from the DB then it cannot know that there is an error. Normally during a write the DB must return information, either an OK or an ERROR; but during a read where DataStage doesn't know how long it might take to complete the application cannot know that something has gone wrong - at least if the processes return no status.
This might have been what happened when the database failed over - the processes are still there but aren't sending any error messages or data back so DataStage just ... waits ...

chulett · Post by **chulett** » Wed Aug 18, 2010 6:02 am

They were still actually running or just in a 'running state'? Could you find active processes for the jobs?

arunpramanik · Post by **arunpramanik** » Wed Aug 18, 2010 6:44 am

chulett wrote:They were still actually running or just in a 'running state'? Could you find active processes for the jobs? ...

Jobs were in Running State. We were not able to stop from Director.
We had to clear the status.

arunpramanik · Post by **arunpramanik** » Wed Aug 18, 2010 6:45 am

ArndW wrote:Was DataStage reading from the database when it was failed over? If DataStage doesn't get an error from the DB then it cannot know that there is an error. Normally during a write the DB must return in ...

That is where I am unable to reason - if DBA has killed the process why the job still in running state, i think job should abort.

ArndW · Post by **ArndW** » Wed Aug 18, 2010 6:58 am

It depends upon the database. In your case, it looks like the job processes were gone but the status wasn't updated accordingly - which is rather a rare occurrence.

arunpramanik · Post by **arunpramanik** » Wed Aug 18, 2010 7:05 am

ArndW wrote: a rare occurrence. ...

I hope so, if this rare occurence is due to absence of connectivity at time process kill.

chulett · Post by **chulett** » Wed Aug 18, 2010 7:20 am

Are you certain that in all that patching / killing that your DataStage server wasn't simply rebooted while your jobs were running?

arunpramanik · Post by **arunpramanik** » Wed Aug 18, 2010 7:39 am

chulett wrote:Are you certain that in all that patching / killing that your DataStage server wasn't simply rebooted while your jobs were running? ...

The patching activities were done on MS SQL server cluster and process that was killed was DB processes. DS server was running and nothing was done on DS server.