bin/uv -admin -start without error but still dsrpc unstarted

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
just4geeks
Premium Member
Premium Member
Posts: 644
Joined: Sat Aug 26, 2006 3:59 pm
Location: Mclean, VA

bin/uv -admin -start without error but still dsrpc unstarted

Post by just4geeks »

I have searched the whole forum but nothing helpful came up hence posting this question.

We are trying to start the DataStage 8.1 on our server using uv -admin -start command but still dsrpc service is not starting.

I get following messages but no error messages:
Starting JobMonApp
JobMonApp has been started.

Any help is appreciated.
Attitude is everything....
dganeshm
Premium Member
Premium Member
Posts: 91
Joined: Tue Aug 11, 2009 3:26 pm

Re: bin/uv -admin -start without error but still dsrpc unsta

Post by dganeshm »

just4geeks wrote:I have searched the whole forum but nothing helpful came up hence posting this question.

We are trying to start the DataStage 8.1 on our server using uv -admin -start command but still dsrpc service is not starting.

I get following messages but no error messages:
Starting JobMonApp
JobMonApp has been started.

Any help is appreciated.

do a ps -ef | grep dsrpc and see if there are any hung processes..
do a netstat -an | grep 315 and send the output
Regards,
Ganesh
just4geeks
Premium Member
Premium Member
Posts: 644
Joined: Sat Aug 26, 2006 3:59 pm
Location: Mclean, VA

Re: bin/uv -admin -start without error but still dsrpc unsta

Post by just4geeks »

dganeshm wrote: do a ps -ef | grep dsrpc and see if there are any hung processes..
do a netstat -an | grep 315 and send the output
There are no hung processes. One thing I would like to mention is we have DS 7.5 as well as DS 8.1 on the same machine. DS 7.5 is using the default port while DS 8.1 uses port 31542.

Code: Select all

: netstat -an | grep 315
tcp4       0      0  159.127.3.148.31538    172.23.242.39.1693     ESTABLISHED
tcp4       0      0  159.127.3.148.31538    172.23.242.7.3315      ESTABLISHED
tcp4       0      0  159.127.3.148.31538    172.23.242.1.3804      ESTABLISHED
tcp4       0      0  159.127.3.148.31538    172.23.242.1.4112      ESTABLISHED
tcp4       0      0  159.127.3.148.31538    172.23.242.35.4270     ESTABLISHED
tcp        0      0  159.127.3.148.50066    159.127.3.148.31531    TIME_WAIT
tcp4       0      0  127.0.0.1.31533        127.0.0.1.64355        ESTABLISHED
tcp        0      0  127.0.0.1.64355        127.0.0.1.31533        ESTABLISHED
tcp        0      0  *.31531                *.*                    LISTEN
tcp        0      0  *.31533                *.*                    LISTEN
tcp4       0      0  *.31538                *.*                    LISTEN
tcp4       0      0  127.0.0.1.31538        127.0.0.1.48959        ESTABLISHED
tcp4       0      0  127.0.0.1.48959        127.0.0.1.31538        ESTABLISHED
tcp4       0      0  127.0.0.1.31538        127.0.0.1.48985        ESTABLISHED
tcp4       0      0  127.0.0.1.48985        127.0.0.1.31538        ESTABLISHED
tcp4       0      0  127.0.0.1.31538        127.0.0.1.48997        ESTABLISHED
tcp4       0      0  127.0.0.1.48997        127.0.0.1.31538        ESTABLISHED
tcp4       0      0  127.0.0.1.31538        127.0.0.1.49146        ESTABLISHED
tcp4       0      0  127.0.0.1.49146        127.0.0.1.31538        ESTABLISHED
tcp4       0      0  127.0.0.1.31538        127.0.0.1.49738        ESTABLISHED
tcp4       0      0  127.0.0.1.49738        127.0.0.1.31538        ESTABLISHED
tcp4       0      0  127.0.0.1.31538        127.0.0.1.49859        ESTABLISHED
tcp4       0      0  127.0.0.1.49859        127.0.0.1.31538        ESTABLISHED
tcp4       0      0  127.0.0.1.31538        127.0.0.1.49860        ESTABLISHED
tcp4       0      0  127.0.0.1.49860        127.0.0.1.31538        ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.42.154.1193    ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.42.154.1230    ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.42.103.1343    ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.42.103.3948    ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.248.31.1268    ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.248.73.1510    ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.248.73.1514    ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.248.31.1856    ESTABLISHED
tcp4       0      0  159.127.3.148.31538    159.127.248.16.2128    ESTABLISHED
Last edited by just4geeks on Thu Mar 25, 2010 12:05 pm, edited 2 times in total.
Attitude is everything....
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

DataStage may not have shut down cleanly. If there are processes still out there waiting on a port after it was shut down, then DataStage will exhibit the symptoms you are describing (no listener after restart).

If you can do so, try the following:

1) Check using the web client (http://servername:9080) to insure that no-one is logged on by checking "Active Sessions". If they are on, ask them to logout and wait for the active session list to only show your web admin session (hit refresh). Then logout of the web client.

2) Go into the Director and make sure there are no jobs active in any project. Stop schedules if you have to.

3) Once you know everyone is out of DataStage completely, check for slave processes on the server (ps -ef | grep dsapi_slave). If you see slave processes still on, those are what is causing your problem. They are leftover processes from crashed connections (network blip, PC crashed, etc.). To get rid of them, very carefully have the administrator issue a "nice kill" (kill -15 pid) on them. This "nice kill" will allow the process to stop semi-cleanly, cleanup any locks and then notifiy its parent client process (will be a "dscs" process if you look for it with ps) to also logout cleanly. Processes should go away almost immediately. If they don't then you may have to do a hard kill (kill -9 pid) on them. You need to insure that all dscs and dsapi_slave processes are off the system.

4) Next shutdown the DataStage engine (bin/uv -admin -stop).

5) Now do a "netstat -a | grep dsrpc" and see if any "listener" processes are stuck on the port. If you are lucky, it will come back "clean" (empty). If you see any processes out there (usually in a FINWAIT state) they are in timeout mode and will go away in about 10 to 15 minutes. The goal of doing step 3 (getting rid of hung processes) is to hopefully get everything off the system semi-cleanly so you don't have to wait 15 minutes.

6) Once the "netstat -a | grep dsrpc" comes back clean (empty) then you can restart DataStage (bin/uv -admin -start) and it should start cleanly.

[* EDIT - Hmm - took a while to type that response, and after posting I noticed you just posted that there are two DataStage instances on this server - DO NOT follow these instructions then unless you plan on shutting down BOTH instances *]
Last edited by asorrell on Thu Mar 25, 2010 11:30 am, edited 1 time in total.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
dganeshm
Premium Member
Premium Member
Posts: 91
Joined: Tue Aug 11, 2009 3:26 pm

Post by dganeshm »

Andy has given almost all the instructions to abide.. if then too it doesnt work..please go in for a complete restart ==engine , asb , websphere engine..
Regards,
Ganesh
Post Reply