Hi,
We are using DataStage on Unix installation. Also the DataStage jobs are used to connect/load data into SAP-BW.
We have following problems encounterd almost everyday.
After certain time of usage in a day, additional users are not able to connect to DataStage Engine. We have total 10 licences.
As soon as the problem is faced,we does the following at UNIX server
connect through dsadm and
$ netstat -a | grep uvrpc
The report shows 5-6 connections as "ESTABLISHED" and many other in FIN_WAIT1, FIN_WAIT2, CLOSE_WAIT etc.
My questions are
1) what are these FIN_WAIT1, FIN_WAIT2, CLOSE_WAIT status in netstat report?
2) Are these active connections OR terminated connections which are stoping me to establish more connections?
3) How to identify process-id for these jobs listed in netstat report so that i can kill those processes from unix prompt?
4) Will killing of these processes will affect any of DataStage jobs?
5) Most important, How do I prevent having such jobs occuring with netstat report status FIN_WAIT1, FIN_WAIT2, CLOSE_WAIT?
Currently we observe the practise of restarting DataStage Engine when we face the problem. However practically we wait for more then 15-20 minutes to automatcially clear all such processes after stopping the DataStage engine.
Please suggest.
Thanks in advance.
Sanjay Desai
DataStage Unix Help
Moderators: chulett, rschirm, roy
-
desais_2001
- Participant
- Posts: 9
- Joined: Fri Feb 28, 2003 3:52 am
- Location: India
Hello
I am assuming that the DataStage user community doesn't do anything nasty with the clients (e.g. power down machines, sever network connection) while logged in to DataStage
For expediency, I'll try to answer your questions in order:
quote:
What are these FIN_WAIT1, FIN_WAIT2, CLOSE_WAIT status in netstat report?
I'm sorry, I don't really know. Someone with more experience in networking shold be able to answer. From what I've encountered, this status is observed on a 'dead' client connection that has not been booted out of either the DataStage server or from the TCP/IP stack.
quote:
Are these active connections OR terminated connections which are stoping me to establish more connections?
They are not active connections. Active connections have a status of ESTABLISHED.
quote:
How to identify process-id for these jobs listed in netstat report so that i can kill those processes from unix prompt?
There are a couple of things you can do in the Administrator to take this further.
a) Check the client time-out setting (General tab)
b) Enable job administration in Director (Project properties, General tab)
The client time-out (in settings) should help manage the 'dropped' connections by logging the sessions out given a set period of inactivity. This doesn't help you debug, but it helps keeping the system clean-ish by regularly discarding idle connections.
Enabling job administration from Director will enable the 'Cleanup Resources' option in the Job menu. In the Cleanup Resources, you are able to see what PIDs DataStage claims to be using. By selecting to display the locks held by process (in the bottom window), you will get some more hints at what a particular PID is doing.
The lotek way would be to telnet to the DataStage server engine. First thing to do is
LISTU. This will give you some dates so you can immediately spot rogue connections (ie ones that are too old). Once the rogue connections have been identified (and written on a piece of paper for later reference), the DS.TOOLS (menu options 5,7) will help clear locks held by a process and log the process off. Make sure the locks get cleared before logging out the process. From memory, the oldest process there is the actual engine - so don't kill that one [:)]
At times, the process may not want to die. kill -9 is useful at this point, followed by another attempt to log the process out of DataStage (if anyone is looking for a Windows equivalent of kill -9, the kill command from the NT resource kit does the trick).
quote:
Will killing of these processes will affect any of DataStage jobs?
No. The client connections don't have anything to do with running jobs. However, one must make sure that the connections to be killed belong the DataStage client connections rather than anything else (ftp, etc)
quote:
Most important, How do I prevent having such jobs occuring with netstat report status FIN_WAIT1, FIN_WAIT2, CLOSE_WAIT
You have to mark the trickiest as most important, eh? [:)] I'm afraid this is beyond my knowledge. Other than observing that my initial assumption is correct, I cannot suggest anything.
I hope this helped,
Mihai
_________________
desk: 01908448571
I am assuming that the DataStage user community doesn't do anything nasty with the clients (e.g. power down machines, sever network connection) while logged in to DataStage
For expediency, I'll try to answer your questions in order:
quote:
What are these FIN_WAIT1, FIN_WAIT2, CLOSE_WAIT status in netstat report?
I'm sorry, I don't really know. Someone with more experience in networking shold be able to answer. From what I've encountered, this status is observed on a 'dead' client connection that has not been booted out of either the DataStage server or from the TCP/IP stack.
quote:
Are these active connections OR terminated connections which are stoping me to establish more connections?
They are not active connections. Active connections have a status of ESTABLISHED.
quote:
How to identify process-id for these jobs listed in netstat report so that i can kill those processes from unix prompt?
There are a couple of things you can do in the Administrator to take this further.
a) Check the client time-out setting (General tab)
b) Enable job administration in Director (Project properties, General tab)
The client time-out (in settings) should help manage the 'dropped' connections by logging the sessions out given a set period of inactivity. This doesn't help you debug, but it helps keeping the system clean-ish by regularly discarding idle connections.
Enabling job administration from Director will enable the 'Cleanup Resources' option in the Job menu. In the Cleanup Resources, you are able to see what PIDs DataStage claims to be using. By selecting to display the locks held by process (in the bottom window), you will get some more hints at what a particular PID is doing.
The lotek way would be to telnet to the DataStage server engine. First thing to do is
LISTU. This will give you some dates so you can immediately spot rogue connections (ie ones that are too old). Once the rogue connections have been identified (and written on a piece of paper for later reference), the DS.TOOLS (menu options 5,7) will help clear locks held by a process and log the process off. Make sure the locks get cleared before logging out the process. From memory, the oldest process there is the actual engine - so don't kill that one [:)]
At times, the process may not want to die. kill -9 is useful at this point
, followed by another attempt to log the process out of DataStage (if anyone is looking for a Windows equivalent of kill -9, the kill command from the NT resource kit does the trick).quote:
Will killing of these processes will affect any of DataStage jobs?
No. The client connections don't have anything to do with running jobs. However, one must make sure that the connections to be killed belong the DataStage client connections rather than anything else (ftp, etc)
quote:
Most important, How do I prevent having such jobs occuring with netstat report status FIN_WAIT1, FIN_WAIT2, CLOSE_WAIT
You have to mark the trickiest as most important, eh? [:)] I'm afraid this is beyond my knowledge. Other than observing that my initial assumption is correct, I cannot suggest anything.
I hope this helped,
Mihai
_________________
desk: 01908448571