Jobs running very longtime

amarnadh0203 · Post by **amarnadh0203** » Tue Jul 29, 2008 8:19 am

Hi All,

We are moving from 7.1R1 server environment to 7.5.2 parallel linux environment.

In first phase we have migrated all our server jobs from 7.1R1 to 7.5.2 and did minimal changes to the jobs ex: environment variables, paths etc.

As part of test we are running the jobs the jobs keeps running more than 24hours where as the same job runs on 7.1R1 in a minute or two.

when i have seen the processes on the linux server "Phantom DSD.RUN" keeps-on running.

all that the jobs does is pull the data from SQL table and load into a sequential file. The job has several individual processes in single server job.

Any help will be useful ... thanks in advance.

Regards,
Amar....

ArndW · Post by **ArndW** » Tue Jul 29, 2008 8:23 am

There are several things I would do. Ask your DBA to see if sessions are open to SQL Server, and check to see if the job is still running as a UNIX process. Are any of these processes using CPU or IO? Use "truss" or your OS's equivalent to see which system calls are being used. That's for starters, there are more options available, but the ones listed above will almost always narrow down the cause.

amarnadh0203 · Post by **amarnadh0203** » Tue Jul 29, 2008 9:43 am

i have seen the processes in the SQL server in "AWATING COMMAND" status for a long time ... i am also seeing processes in Linux server. Job remains in the Running status too.

n431c7 20823 26610 0 11:36 ? 00:00:00 phantom DSD.RUN P444_04_RI_Control_Stat 0/0/1/0/0
n431c7 20832 20823 0 11:36 ? 00:00:00 phantom DSD.StageRun P444_04_RI_Control_Stat. P444_04_RI_Control_Stat.Transformer_3 1 0/0/1
n431c7 20836 20823 0 11:36 ? 00:00:00 phantom DSD.StageRun P444_04_RI_Control_Stat. P444_04_RI_Control_Stat.Transformer_7 1 0/0/1
n431c7 20837 20823 0 11:36 ? 00:00:00 phantom DSD.StageRun P444_04_RI_Control_Stat. P444_04_RI_Control_Stat.Transformer_8 1 0/0/1
n431c7 20838 20823 0 11:36 ? 00:00:00 phantom DSD.StageRun P444_04_RI_Control_Stat. P444_04_RI_Control_Stat.Transfor

ArndW · Post by **ArndW** » Tue Jul 29, 2008 9:55 am

What does "truss -p 20832" show?

amarnadh0203 · Post by **amarnadh0203** » Tue Jul 29, 2008 10:01 am

"truss -p 20832" it days no such command.

Please provide any alternative.

ArndW · Post by **ArndW** » Tue Jul 29, 2008 11:38 am

I went to Google and entered "LINUX equivalent of truss" and found out that you need to enter the "strace" or "ltrace" command.

amarnadh0203 · Post by **amarnadh0203** » Tue Jul 29, 2008 11:46 am

i got the following listed:

n431c7@tedshdu2 /home/n431c7 $ strace -p 18240
Process 18240 attached - interrupt to quit
[ Process PID=18240 runs in 32 bit mode. ]
futex(0x3dc820, FUTEX_WAIT, 2, NULL

ArndW · Post by **ArndW** » Tue Jul 29, 2008 12:00 pm

are the other processes also waiting on the same or similar locks?

amarnadh0203 · Post by **amarnadh0203** » Tue Jul 29, 2008 12:16 pm

Yes, the other processes are also giving the same details as of first one.

Could you please explain me in details...?

satish_valavala · Post by **satish_valavala** » Tue Jul 29, 2008 9:48 pm

Which user compiled the jobs? and which user is running jobs? See if you have any user group (unix) level permission issues.

ray.wurlod · Post by **ray.wurlod** » Tue Jul 29, 2008 11:29 pm

Are these server jobs (as marked) or parallel jobs (as posted)?

ArndW · Post by **ArndW** » Wed Jul 30, 2008 2:31 am

I haven't worked with FUTEX locks, but it seems to be the LINUX equivalent of MUTEX calls. So all the separate server job processes seem to be waiting on a signal that they are probably never going to get. Do you have any "defunct" or "zombie" processes visible?