SFTP Script

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
pradkumar
Charter Member
Charter Member
Posts: 393
Joined: Wed Oct 18, 2006 1:09 pm

SFTP Script

Post by pradkumar »

Hi Everyone,

Below is the script which is used to get files from FTP share site.

***********************Script*************
# name: GetFile.ksh
#
# purpose:
#
# Script to get files from FTP share site.
# One control file is used
#
# /ds/etc/FTPPassword.ctl (text file in secured location)
#
# revisions:
#
# 11-18-08, vvivek, Initial Version
# -----------------------------------------------------------------------------
# set environment
# -----------------------------------------------------------------------------
set -vx
SetEnv()
{
logDir=/export/home/vvivek
PasswordCtl=/export/home/vvivek/FTPPassword.ctl
FTPInDir=/export/home/vvivek
emailGroup=vvivek@amtech.com
}

FTPFile()
{
rcFTP=0

\echo "Receiving $FileName from ftphost ($ServerName)..."
\ftp -inv <<EOF > $ftpLog
open $ServerName
user "$Ftpusername" $Ftppassword
lcd $Ftplocaldir
get $DataFile $FileName
quit
EOF
#--------------------------------------------------------------
# checking for the errors during FTP process
#--------------------------------------------------------------
if [[ -s $ftpLog ]]; then
if [[ -n $(\grep "^226 Transfer complete." $ftpLog) ]]; then
rcFTP=0
else
rcFTP=1
fi
else
\echo "****** unknown error with FTP while getting file ($FileName)! ********"
rcFTP=1
fi
}

# -----------------------------------------------------------------------------
# main
# -----------------------------------------------------------------------------

Me=$(\echo $0 | \awk -F"/" '{print $NF}')
mypath=$(\echo $0 | \awk -F"$Me" '{print $1}')
SetEnv
pid=$$
logFile=$logDir/$(\echo $Me | \awk -F. '{print $1}').log.$(\date +%Y%m%d)
exec 1>$logFile
exec 2>&1
ftpLog=$logDir/$(\echo $Me | \awk -F. '{print $1}')_FTPLog_$pid.log
tmpFile=$logDir/$(\echo $Me | \awk -F. '{print $1}').tmp

if [ ! -f $PasswordCtl ]
then
echo "******** Password file $PasswordCtl not found. Aborting the job ************"
subject="$Me FAILED - Password file not found"
print 'Pls check the attached log file \n\nThanks' >$tmpFile
mutt -a $logFile -s "$subject" $emailGroup < $tmpFile
exit 1
fi

#-------------------------------------------------------
# Reading parameters from Password.ctl file
#-------------------------------------------------------
ServerName=`grep '^FTPIn1' $PasswordCtl | cut -f2 -d'|'`
Ftpusername=`grep '^FTPIn1' $PasswordCtl | cut -f3 -d'|'`
Ftppassword=`grep '^FTPIn1' $PasswordCtl | cut -f4 -d'|'`
DataFile=`grep '^FTPIn1' $PasswordCtl | cut -f5 -d'|'`
FileName=$(\echo $DataFile | \awk -F"/" '{print $NF}')
Ftplocaldir="$FTPInDir"
#-------------------------------------------------------------
# Transferring the data file
#-------------------------------------------------------------
FTPFile

if [[ $rcFTP != 0 ]]; then
echo "********* unable to get the file ($FileName) from ftphost ($ServerName)! *********"
subject="$Me FAILED - FTP of file failed"
print 'Pls check the attached FTP log for errors \n\nThanks' >$tmpFile
mutt -a $logFile -a $ftpLog -s "$subject" $emailGroup < $tmpFile
exit 1
fi

echo "File $FileName successfully transferred from $ServerName at $(\date '+%m-%d-%y %H:%M:%S')"

exit 0


The script is working fine.

But as per client standards they have to use SFTP to transfer the files.

Even for SFTP also they have given me the hostname , username and the password.

How to change the FTP script to SFTP and how to automate this script . since i tried to login using a sample script.

#!/usr/bin/ksh
sftp userid@hostname << EOF
lcd /export/home/vvivek
get TEST.TXT
quit
EOF

I am having a problem doing this because no matter what I do, when I run the above script, I get a prompt at command line asking for a password. How could I automate this?

Could someone suggest what changes have to make in the script. For the FTP iam using Password ctl file to pass the host credentials.

Thanks in Advance
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Whenever you get a prompt at command line asking for a password using the sftp, that means the public key (ssh) between the sending and receiving servers is not set up yet.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You can't automate secure anything to take a password, that's just not how it works. Google for "passwordless sftp". Basicaly, you need to create an RSA Key file and install it on the target server so it knows you are ok to let in the door.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply