SCP problem (Command Stage)

metabill · Post by **metabill** » Fri Aug 12, 2005 8:01 am

I'm trying to use a Command Stage to run scp to securely retrieve some CSV files from linux box so that they can be processed in a job. I've generated a private/public key pair so that a password will not have to be supplied, and can successfully run the scp command from a windows command prompt outside DataStage and retrieve the files. (The command I'm using is actually pscp, from the PuTTY folks). When I run the job within the Command Stage, however, the command returns an error: "The server's host key is not cached in the registry....." and the connection attempt is abandoned.

I suspect the problem is that the windows user account under which the DataStage engine is being run has not previously been used to establish an SSH connection with the linux box. As I recall, the first time such a connection is made, you are prompted as to whether or not you want to accept the server's host key. I think that once you accept the key, you are no longer prompted. If this suspicion is correct, I think I need to manually make the connection once using the DataStage user account, then all will (hopefully) be well. Or something like that.

Thanks in advance for any help with this!

metabill · Post by **metabill** » Fri Aug 12, 2005 8:34 am

I should have been more clear about my question: Can you please tell me how to figure out the user account that the DataStage server will be running under? If I can determine that, I will try to run the scp command manually using that account in an attempt to get the server's key cached in the registry.

Thanks.

kduke · Post by **kduke** » Fri Aug 12, 2005 9:01 am

You can always shell out and run pwd. The best way is @WHO will give the Project namd. The path to the project is located in UV.ACCOUNT.

LIST DICT UV.ACCOUNT

metabill · Post by **metabill** » Fri Aug 12, 2005 9:25 am

Thanks for the suggestion. This sounds like it will provide information about the Universe account and environment that will be in use, is that right? If so, I'm guessing that that will not necessarily be the same as the windows user account under which the DataStage server will be running. Please correct me if I have misunderstood.

I was able to locate a running windows service (using Task Manager) named "dsapi_server", which I found out is a DataStage server process of some sort. It is apparently running under the SYSTEM user account. I'm thinking therefore that when I run a DS job that contains a Command Stage, there is a good chance it will run the OS command as the same SYSTEM user.

kduke · Post by **kduke** » Fri Aug 12, 2005 9:41 am

Yes it will run under the user you start the job as. When you run a Command stage then it starts in the directory of the project. If you run a pwd command it should tell you what directory it is attached to. Look in the log. The output of your command should be in there. Make you command echo results.

metabill · Post by **metabill** » Fri Aug 12, 2005 9:58 am

Thanks for the suggestions. Indeed, I can now see the directory (using pwd) and user (using whoami) that is being used to run the command. But when the pscp command is run using the command stage, it is still asking for interactive confirmation to accept the server host key. This is happening even though when I run the same pscp command line outside of datastage as the same user, it uses the server host key previously cached in the registry and not prompting for conformation. This leads me to believe that something is still different when pscp is run through the command stage; I just need to figure out what it is.

metabill · Post by **metabill** » Fri Aug 12, 2005 10:58 am

FYI: I solved the problem, by adding some registry keys/values so that they would apply to the command (pscp) when run in the command stage. In this case, I added the same key structure for the SSH software (to define the cached server host keys) under HKEY_USERS that existed under HKEY_CURRENT_USER. As it turned out, when the command stage runs a command, it is simply not the same user and/or environment as when you run the same command from the windows command line. By adding the needed keys/values under HKEYS_USERS, it now seems to apply to all users and environments, including whatever the DataStage server is using at runtime.

The net result of all this is that my job can now run an scp command (pscp) within the command stage to securely transfer files to the DataStage server, using public/private keys without specifying a password or requiring any other interactive information. Seems like a good thing.

kduke · Post by **kduke** » Fri Aug 12, 2005 2:05 pm

Sounds cool.