we're working in a solaris environment, where datastage jobs are scheduled trough autosys. That means we have to create a short unix script for every workflow we schedule. At this point we've created a runds script which is a wrapper around somedsjob commands . (most of this is indeed info from other posts on this forum)
Code: Select all
#! /bin/ksh
#set -x
#===============================================================================
# script rundsjob.sh
# purpose runs the datastage-job supplied in the parameters
# author
# date
# parameters -S<ds-server>
# -U<ds-user>
# -P<ds-pasword>
# -E<ds-project/environment>
# -J<ds-job>
# -R<reset_flag>
# -A<automaticly-completeprojectflag>
# [-O<ds-parameters>]
# [-I<invocationid>]
#===============================================================================
# changed at
# reason
# changed by
#===============================================================================
# Initialization: execute profiles
#===============================================================================
. /start/lib/rts/common/bin/set_env.sh
. /${teleEnv}/lib/sft/dts/bin/dts_profile.sh
#------------------------------------------------------------------------
# function : translate ds-returncodes to unix-standards
#---------------------------------------------------------------------------
TransDsToUnix(){
out=3
dscode=${1}
case ${dscode} in
0) out=3 ;; #Job still running
1) out=0 ;; #run ok
2) out=1 ;; #run with warnings
3) out=3 ;; #run failed
21) out=3 ;; #run reset
97) out=3 ;; #run stopped
90) out=3 ;; #start job didn't start requested job
93) out=3 ;; #job started by wStart failed
99) out=3 ;; #job has never run
esac
print ${out}
}
#############################################################################
#start of script
###############################################################################
#----------------------------------------------------------------------------
# check input parameters
#----------------------------------------------------------------------------
filledparm=0
PARAMS_GIVEN='N'
INVOC_GIVEN='N'
while getopts "S:U:P:E:J:R:A:O:I:" option
do
case ${option} in
"U" ) DS_USER=${OPTARG}
((filledparm = $filledparm +1 ))
;;
"S" ) DS_SERVER=${OPTARG}
((filledparm = $filledparm +1 ))
;;
"P" ) DS_PASWRD=${OPTARG}
((filledparm = $filledparm +1 ))
;;
"E" ) DS_PROJECT=${OPTARG}
((filledparm = $filledparm +1 ))
;;
"J" ) DS_JOB=${OPTARG}
((filledparm = $filledparm +1 ))
;;
"R" ) RESET_FLAG=${OPTARG}
((filledparm = $filledparm +1 ))
;;
"A" ) AUT_ADD=${OPTARG}
((filledparm = $filledparm +1 ))
;;
"O" ) DS_OPTIONS=${OPTARG}
PARAMS_GIVEN='Y'
;;
"I" ) INVOC_ID=${OPTARG}
INVOC_GIVEN='Y'
;;
\? ) print "usage : ${0} -S<ds-server> -U<ds-user> -P<ds-pasword> -E<ds-project/environment> -J<ds-job> -R<reset_flag> -A<automaticly-completep
rojectflag> [-O<ds-parameters>] [-I<invocationid>]"
return 4
;;
esac
done
if [[ ${filledparm} < 7 ]] then
print "usage : ${0} -S<ds-server> -U<ds-user> -P<ds-pasword> -E<ds-project/environment> -J<ds-job> -R<reset_flag> -A<automaticly-completeprojectfl
ag> [-O<ds-parameters>] [-I<invocationid>]"
return 4
fi
#-------------------------------------------------------------------
#extra parameter action
#if the projectname has a structure like NAME(ENV) the environment can be automatically added ex PRDTNG
#------------------------------------------------------------------
typeset -u TNENV=${teleEnv}
if [[ ${AUT_ADD} = 'Y' ]] then
DS_PROJECT=${DS_PROJECT}${TNENV}
fi
#--------------------------------------------------------------------
#log what the script tries to start
#-------------------------------------------------------------------
echo trying to run ${DS_JOB} in project ${DS_PROJECT} on server ${DS_SERVER} for user ${DS_USER} with resetflag=${RESET_FLAG}
echo with ds-parameters: ${DS_OPTIONS} PARAMS_GIVEN=${PARAMS_GIVEN}
echo with invocationid: ${INVOC_ID} INVOCATION_GIVEN=${INVOC_GIVEN}
#--------------------------------------------------
#to be checked: can we first test if user and pasword are correct, if they are wrong the messsage below does not fit
#-----------------------------------------------------
#-------------------------------------------------------------------
#check if job exists
#-------------------------------------------------------------------
jobcheck=`${DSHOME}/bin/dsjob -server ${DS_SERVER} -user ${DS_USER} -password ${DS_PASWRD} -ljobs ${DS_PROJECT}|grep ${DS_JOB}`
echo job=$jobcheck
if [[ ${jobcheck} != ${DS_JOB} ]] then
print "job: " ${DS_JOB} " does not exist in project:"${DS_PROJECT}
return 4
fi
#------------------------------------------------------------------
#check current status of this job
#if resetflag =Y perform reset when not ready, else abort when status invalid to run the job
#-----------------------------------------------------------------
CUR_STATUS=`${DSHOME}/bin/dsjob -server ${DS_SERVER} -user ${DS_USER} -password ${DS_PASWRD} -jobinfo ${DS_PROJECT} ${DS_JOB}|grep 'Job Status'|cut -d
'(' -f2|cut -d ')' -f1`
echo current status=${CUR_STATUS}
if [[ ${CUR_STATUS} = 1 ]] then #previous run succesfull
echo jobstatus before run ok, continue
elif [[ ${CUR_STATUS} = 2 ]] then #previous run succesfull with warnings
echo jobstatus before run ok, continue
elif [[ ${CUR_STATUS} = 99 ]] then #first run for this job after compile
echo jobstatus before run ok, continue
elif [[ ${CUR_STATUS} = 3 ]] then #previous run aborted
if [[ ${RESET_FLAG} = 'Y' ]] then
#reset job
echo previous run failed performing automatic reset
${DSHOME}/bin/dsjob -server ${DS_SERVER} -user ${DS_USER} -password ${DS_PASWRD} -run -mode RESET ${DS_PROJECT} ${DS_JOB}
else echo previous run aborted, no automatic reset specified stop with error
return 4 #error
fi
else echo not sure about state current status: ${CUR_STATUS} lets try to start
fi
#-------------------------------------------------------------------------
#actually run the job with or without params
#------------------------------------------------------------------------
if [[ ${PARAMS_GIVEN} = 'N' ]] then
if [[ ${INVOC_GIVEN} = 'N' ]] then
echo starting job without parameters and without invocationid
${DSHOME}/bin/dsjob -server ${DS_SERVER} -user ${DS_USER} -password ${DS_PASWRD} -run -wait -jobstatus ${DS_PROJECT} ${DS_JOB}
dsretcod=$?
else echo starting job without parameters and with invocationid
${DSHOME}/bin/dsjob -server ${DS_SERVER} -user ${DS_USER} -password ${DS_PASWRD} -run -wait -jobstatus ${DS_PROJECT} ${DS_JOB}.${INVOC_ID}
dsretcod=$?
fi
else
if [[ ${INVOC_GIVEN} = 'N' ]] then
echo starting job with parameters and without invocationid
${DSHOME}/bin/dsjob -server ${DS_SERVER} -user ${DS_USER} -password ${DS_PASWRD} -run -param ${DS_OPTIONS} -wait -jobstatus ${DS_PROJECT} ${DS_JO
B}
dsretcod=$?
else echo starting job with parameters and with invocationid
${DSHOME}/bin/dsjob -server ${DS_SERVER} -user ${DS_USER} -password ${DS_PASWRD} -run -param ${DS_OPTIONS} -wait -jobstatus ${DS_PROJECT} ${DS_
JOB}.${INVOC_ID}
dsretcod=$?
fi
fi
echo returncode from datastage is ${dsretcod}
#-----------------------------------------------------------------------
#translate output to standards
#translate dsoutput to 3values:succes,warnings,fail
#wStart is an overall job that starts other jobs following some standards. wStart itself will never abort, real status needs to be taken from the log
#----------------------------------------------------------------------
if [[ ${DS_JOB} = 'wStart' ]] then #for now this check is only done if job is wStart
echo since job=${DS_JOB} extra check on log is performed
dsretcodlog=`${DSHOME}/bin/dsjob -server ${DS_SERVER} -user ${DS_USER} -password ${DS_PASWRD} -logsum -type INFO -max 5 ${DS_PROJECT} ${DS_JOB}
|grep '@@JOBSTATUS'|cut -d '=' -f2|cut -d',' -f1`
echo returncode from Log=${dsretcodlog}
fi
if [[ ${dsretcodlog} -gt ${dsretcod} ]] then
echo updating returncode to logvalue
dsretcod=${dsretcodlog}
fi
U_RETURN=`TransDsToUnix ${dsretcod}`
echo datastage returncode translated to unixcode is ${U_RETURN}
return $U_RETURN
I still have 2 problems with this script
1: paswords are easily obtained while this script is running. I understand that there is no workaround for this. Anyone any idea if there would be improvement in upcoming versions?
2: we start of by checking if the job exists before running it. But we should actually first check if user and pasword are correct. If an incorrect user is now supplied this script will return that the job does not exist. Is there any way this can be done: just log on to check if user is valid for this server/project?
