DataStage job stuck in a running state

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
orondedata
Participant
Posts: 1
Joined: Fri Nov 06, 2009 3:15 pm
Location: Cincinnati

DataStage job stuck in a running state

Post by orondedata »

I have been support DataStage for 2 weeks now and I have had a couple jobs get stuck in a running state. What is the best automated way to monitor job that never complete since they don't send a success or failure email.
Oronde Williams
CBIP, Data Analysis and Design
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

It's important to understand that the status of a job is basically just a 'flag', something that gets set when the job starts and adjusted when the job finishes. So server crashes or unusual situations where a job may just core dump and fall over dead can leave it stuck forever in a 'Running' status.

First thing to do is check to see if it is actually running or not. From the UNIX command line issue a "ps -ef | grep phantom" and see if you can see any processes associated with the job name in question. If nothing shows up, then you can use the Director client to reset that status flag. Go to the Job menu and choose Clear Status File. It will ask twice if you're really sure you want to do this as it is not recommended to do this to a job that is actually running, but after you say 'yes' it will set the job's status back to Compiled and you can go from there.
-craig

"You can never have too many knives" -- Logan Nine Fingers
austin_316
Participant
Posts: 80
Joined: Fri Aug 21, 2009 7:49 am
Location: India

DataStage job stuck in a running state

Post by austin_316 »

one of my job is getting stuck and i have given the command "ps -ef | grep phantom" in the unix. i am able to see a process assocaited with my job. how can i stop that job! please help me in this.
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Did you try the stop button in director ?

Are you sure that the process is running but not doing anything ?

Is it in any deadlock situation or doing an after-job script or waiting for something ?

I will recommend checking the process action before killing or stopping it.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Check for lock contention in any database to which the job connects. A job will wait up to forever for a database lock (or table or row lock) to be released.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Sreenivasulu
Premium Member
Premium Member
Posts: 892
Joined: Thu Oct 16, 2003 5:18 am

Post by Sreenivasulu »

Ray is correct. This happened for me last week and the issue was that the 'database was not in right shape'. May be for you it might be some other issue.
But first
1.you need to stop the long-running job
2.If it has run earlier properly then run-again
3.If step 2 fails check the enviroment

Regards
Sreeni
austin_316
Participant
Posts: 80
Joined: Fri Aug 21, 2009 7:49 am
Location: India

DataStage job stuck in a running state

Post by austin_316 »

Yes as suggested by Ray the issue was with some lock on a table. i have modified the rows of the table manually and forgot to commit the db. there was a before job script which is trying to update hte same row in the same table. but since the lock was already made on the row the job was waiting for it to be released. when i commited the db then the process continued.
This solution i got for one situation.but again when again the job was in runnning condition for a long time i tried the same process. but still the problem persisted. so may there was some other reason for this. Not able to find it though :?[/code]
austin_316
Participant
Posts: 80
Joined: Fri Aug 21, 2009 7:49 am
Location: India

DataStage job stuck in a running state

Post by austin_316 »

Yes as suggested by Ray the issue was with some lock on a table. i have modified the rows of the table manually and forgot to commit the db. there was a before job script which is trying to update hte same row in the same table. but since the lock was already made on the row the job was waiting for it to be released. when i commited the db then the process continued.
This solution i got for one situation.but again when again the job was in runnning condition for a long time i tried the same process. but still the problem persisted. so may there was some other reason for this. Not able to find it though :?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Have your DBA check for deadlocks and lock waits in the database.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply