Job geting hanged

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ramank1981
Participant
Posts: 10
Joined: Wed Dec 21, 2005 5:03 am

Job geting hanged

Post by ramank1981 »

Hi ,
I am facing a strange problem.
I am running one job with sequential file as source and Db2 as target with some transformations in between.

Problem is that sometimes the job runs fine and populates data in target table, but on other day the same job gets hanged and keeps on runing until i go and stop from the director.

We are facing this isse since long time.
Ultramundane
Participant
Posts: 407
Joined: Mon Jun 27, 2005 8:54 am
Location: Walker, Michigan
Contact:

Re: Job geting hanged

Post by Ultramundane »

ramank1981 wrote:Hi ,
I am facing a strange problem.
I am running one job with sequential file as source and Db2 as target with some transformations in between.

Problem is that sometimes the job runs fine and populates data in target table, but on other day the same job gets hanged and keeps on runing until i go and stop from the director.

We are facing this isse since long time.
What OS and version? Did you make sure that nothing was hung in DB2? What step was the job on when it hung?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Did you make sure that nothing was locked in DB2?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ramank1981
Participant
Posts: 10
Joined: Wed Dec 21, 2005 5:03 am

Post by ramank1981 »

I checked the DB2 stage, there is no hung process of DB2.
Stange part is that we ran the same job with same source , same target 2 days back and the job was completed in few seconds.
But now the job is hanging ...
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Wrap your job design in

Code: Select all

 tags so we can see it properly.

Add Peek stages on relevant links so you can truly verify where data are and are not flowing.

And you need to check the DB2 database (not the DB2 stage) to determine whether anything is locked.  Are you, for example, updating one or both of the tables that provide input to the Join stage?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Ultramundane
Participant
Posts: 407
Joined: Mon Jun 27, 2005 8:54 am
Location: Walker, Michigan
Contact:

Post by Ultramundane »

ramank1981 wrote:I checked the DB2 stage, there is no hung process of DB2.
Stange part is that we ran the same job with same source , same target 2 days back and the job was completed in few seconds.
But now the job is hanging ...
What OS and version? Just curious, because AIX has a patch to fix fifo thread intermittent hangs caused by a timing issue. This caused us big problems.
rajendharreddy
Participant
Posts: 46
Joined: Tue Aug 21, 2007 5:39 am
Location: Bangalore, India

Post by rajendharreddy »

One thing you can try is, drop the table and recreate the table with proper table space. Thanks..
dspxguy
Participant
Posts: 156
Joined: Thu May 24, 2007 4:09 pm
Location: Simi Valley, CA

Post by dspxguy »

Ultramundane wrote:
ramank1981 wrote:I checked the DB2 stage, there is no hung process of DB2.
Stange part is that we ran the same job with same source , same target 2 days back and the job was completed in few seconds.
But now the job is hanging ...
What OS and version? Just curious, because AIX has a patch to fix fifo thread intermittent hangs caused by a timing issue. This caused us big problems.
Hi, can you please explain the problems you faced and how did you resolve them, what version and o/s you have?
Please let me know, Thanks
ramank1981
Participant
Posts: 10
Joined: Wed Dec 21, 2005 5:03 am

Problem is still there..not resolved.

Post by ramank1981 »

we are using AIX 5.3. it still seems like a resource problem, no further clue.
Raghavendra
Participant
Posts: 147
Joined: Sat Apr 30, 2005 1:23 am
Location: Bangalore,India

Post by Raghavendra »

can you please check if your /tmp directory is filled.
And also check for &PH& directory.
Ultramundane
Participant
Posts: 407
Joined: Mon Jun 27, 2005 8:54 am
Location: Walker, Michigan
Contact:

Re: Problem is still there..not resolved.

Post by Ultramundane »

ramank1981 wrote:we are using AIX 5.3. it still seems like a resource problem, no further clue.
You could be running into the fifo_open problem. This caused us a lot of job hangs. There is patch for the AIX kernel to address this issue. The patch is: IY86253

I think you can google the patch.
ramank1981
Participant
Posts: 10
Joined: Wed Dec 21, 2005 5:03 am

Post by ramank1981 »

I had earlier cleared the &PH& , but the job was still hanging.
I have asked AIX admin for the patch IY86253, waiting for him to respond.
ramank1981
Participant
Posts: 10
Joined: Wed Dec 21, 2005 5:03 am

Post by ramank1981 »

It seems we finally found the root cause of the problem. One of the routine was creating the bottle neck. After we implemented the same thing with differnent logic, the jobs are running fine. :D

Thanks to all of you for giving inputs.
Post Reply