Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.
Moderators: chulett , rschirm , roy
avi21st
Charter Member
Posts: 135 Joined: Thu May 26, 2005 10:21 am
Location: USA
Post
by avi21st » Thu Aug 03, 2006 10:24 am
Hi
I have created a shell script to Execute Datastage job but I am getting thos error while executing it.
Code: Select all
Could not load program /appl/ascential/Ascential/DataStage/DSEngine/bin/dsjob:
Dependent module libvmdsapi.so could not be loaded.
Could not load module libvmdsapi.so.
Please suggest what to do on my part.
This is the code which I have used:
Code: Select all
#!/bin/ksh
############################################################################
# Program : DSExecuteJob.ksh
#
# Input : Datastage Job Name, Datastage Project Name
#
#
# Output Parameters: 0 - For Success
# Error Description - For failure
#
# Developer : Avishek
#
# Date : August-03-2006
#
# Modification History
# Version Modified by Date Modified Modification
# --------------------------------------------------------------------------
############################################################################
############################################################################
#
# Store Input Parameters in Shell Variables
#
############################################################################
FILE_NAME=`basename $0 .ksh`
DSJOBNAME=$1
DSPROJNAME=$2
DSSERVER="schdcbss01"
DSUSERID="mukherav"
DSPASSWORD="Infy@2003"
BinFileDirectory=`cat /.dshome`/bin
PARAMLIST=0
DSLOG="DS_"$DSJOBNAME
echo $DSJOBNAME
echo $DSPROJNAME
echo $DSSERVER
echo $DSUSERID
echo $DSPASSWORD
echo $BinFileDirectory
echo $PARAMLIST
echo $DSLOG
DSLOG_FILE=/WSI/ETL/Conversion/LogFiles/Misc/$DSLOG.log
echo $DSLOG_FILE
rm $DSLOG_FILE
DT_START=`date`
echo "-----------------------------------------" >> $DSLOG_FILE
echo "Execution Started at for Datastage Job Sequencer:" $DSJOBNAME " on date:"$DT_START>>$DSLOG_FILE
echo "Datastage Project:" $DSPROJNAME >>$DSLOG_FILE
############################################################################
#
# DataStage EXECUTION of the job sequencer
#
############################################################################
CURR_DT=`date`
echo "$CURR_DT Executing DataStage dsjob program..." >> $DSLOG_FILE
echo $BinFileDirectory/dsjob -server $DSSERVER -user $DSUSERID -password $DSPASSWORD -run -wait $PARAMLIST $DSPROJNAME $DSJOBNAME 2>&1 > $DSLOG_FILE
echo
echo $BinFileDirectory/dsjob -server $DSSERVER -user $DSUSERID -password $DSPASSWORD -run -wait $PARAMLIST $DSPROJNAME $DSJOBNAME 2>&1 > $DSLOG_FILE
echo
eval $BinFileDirectory/dsjob -server $DSSERVER -user $DSUSERID -password $DSPASSWORD -run -wait $PARAMLIST $DSPROJNAME $DSJOBNAME 2>&1 >> $DSLOG_FILE
jobwaiting=`grep "Waiting for job..." $DSLOG_FILE`
if [ "$jobwaiting" != "Waiting for job..." ]; then
CURR_DT=`date`
echo $CURR_DT $DSJOBNAME "DataStage failed to start the job" >> $DSLOG_FILE
failedstart=1
else
CURR_DT=`date`
echo $CURR_DT $DSJOBNAME "DataStage successfully started the job">> $DSLOG_FILE
failedstart=0
fi
CURR_DT=`date`
echo $CURR_DT $DSJOBNAME "Retrieving job information">> $DSLOG_FILE
$BinFileDirectory/dsjob -server $DSSERVER -user $DSUSERID -password $DSPASSWORD -jobinfo $PARAMLIST $DSPROJNAME $DSJOBNAME >> $DSLOG_FILE
############################################################################
#
# Check Status for Job Execution
#
############################################################################
ERROR=`grep "ERROR" $DSLOG_FILE`
if [ "$failedstart" != 0 ]; then
CURR_DT=`date`
echo $CURR_DT $DSJOBNAME "The job failed to start" >> $DSLOG_FILE
AuditStatus="FAILURE"
Comments="MasterControl aborted"
EXIT_STATUS=1
else
if [ "$ERROR" = 1 -o "$ERROR" = 2 ]; then
CURR_DT=`date`
echo $CURR_DT $DSJOBNAME "The job completed successfully" >> $DSLOG_FILE
AuditStatus="SUCCESS"
Comments=""
EXIT_STATUS=0
else
CURR_DT=`date`
echo $CURR_DT $DSJOBNAME "The job aborted" >> $DSLOG_FILE
AuditStatus="FAILURE"
Comments="MasterControl aborted"
EXIT_STATUS=1
fi
fi
if [ ${EXIT_STATUS} -ne 0 ]
then
echo "Error in the Shell for calling the Datastage job sequencer:" $DSJOBNAME >>$DSLOG_FILE
else
echo "Successfully executed the Shell for calling the Datastage job sequencer:" $DSJOBNAME >>$DSLOG_FILE
fi
DT_END=`date`
echo "Execution Ended with Shell Status as :"$EXIT_STATUS>>$DSLOG_FILE
echo "--------------------------------------------">>$DSLOG_FILE
exit ${EXIT_STATUS}
Avishek Mukherjee
Data Integration Architect
Chicago, IL, USA.
ray.wurlod
Participant
Posts: 54607 Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:
Post
by ray.wurlod » Thu Aug 03, 2006 10:50 am
You need to make sure that the requisite libraries are installed on the machine on which you want to execute the dsjob command and that they are findable via the shared library search list (the LD_LIBRARY_PATH, SHLIB_PATH or LIBPATH environment variable, depending on exactly which UNIX platform you are executing).
Read the chapter on the DataStage Development Kit (Job Control Interfaces) in either the Server Job Developer's Guide or the Parallel Job Advanced Developer's Guide to learn more about redistributing DSAPI applications such as dsjob command.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kcbland
Participant
Posts: 5208 Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:
Post
by kcbland » Thu Aug 03, 2006 11:14 am
Make sure your source your dsenv file as part of your script.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
avi21st
Charter Member
Posts: 135 Joined: Thu May 26, 2005 10:21 am
Location: USA
Post
by avi21st » Thu Aug 03, 2006 2:31 pm
kcbland wrote: Make sure your source your dsenv file as part of your script.
Thanks a lot- Ray and Bland- I fixed the code.
I modified the code a bit and made it much simpler. I am mailing the scripts,
please suggest some improvements . It might help others too.
Code: Select all
#!/bin/ksh
############################################################################
# Program : DSExecuteJob.ksh
#
# Input : Datastage Job Name, Datastage Project Name
#
#
# Output Parameters: 0 - For Success
# Error Description - For failure
#
# Developer : CTS
#
# Date : August-03-2006
#
# Modification History
# Version Modified by Date Modified Modification
# --------------------------------------------------------------------------
############################################################################
############################################################################
#
# Store Input Parameters in Shell Variables
#
############################################################################
FILE_NAME=`basename $0 .ksh`
DSJOBNAME=$1
DSPROJNAME=$2
BinFileDirectory=`cat /.dshome`/bin
DSHOME="/opt/ds/Ascential/DataStage/DSEngine"
DSJOBPATH="/opt/ds/Ascential/DataStage"
DSLOG="DS_"$DSJOBNAME
DSLOG_FILE=/WSI/ETL/Conversion/LogFiles/Misc/$DSLOG.log
rm $DSLOG_FILE
############################################################################
#
# Displaying the parameters in log file
#
############################################################################
echo "Log File Name: "$DSLOG_FILE >> $DSLOG_FILE
echo "Datastage Job to Run:"$DSJOBNAME >> $DSLOG_FILE
echo "Datastage Project: "$DSPROJNAME >> $DSLOG_FILE
echo "Datastage Bin: "$BinFileDirectory >> $DSLOG_FILE
cd $DSHOME
. ./dsenv
cd $DSJOBPATH
############################################################################
#
# DataStage EXECUTION of the job sequencer
#
############################################################################
DT_START=`date`
echo "-----------------------------------------">>$DSLOG_FILE
echo "Execution Started at for Datastage Job Sequencer:" $DSJOBNAME " on date:"$DT_START>>$DSLOG_FILE
################################################################################
# Check old job status here
################################################################################
JOB_STATUS=`${BinFileDirectory}/dsjob -jobinfo $DSPROJNAME $DSJOBNAME | head -1 | cut -d"(" -f2 | cut -d")" -f1`
echo "Before run JOB_STATUS=$JOB_STATUS"
case $JOB_STATUS in
################################################################################
# 0 "Running"
################################################################################
0)
echo "Job $DSJOBNAME already running.Job run Failed." >> $DSLOG_FILE;
exit 1
;;
################################################################################
# Runnable Job Status (do something)
# 1 "Finished"
# 2 "Finished with Warning (see log)"
# 9 "Has been reset"
# 11 "Validated OK"
# 12 "Validated (see log)"
# 21 "Has been reset"
# 99 "Compiled"
################################################################################
1|2|7|9|11|12|21|99)
:
;;
################################################################################
# NOT Runnable Job Status (reset job)
# 0 "Running"
# 3 "Aborted"
# 8 "Failed validation"
# 13 "Failed validation"
# 96 "Aborted"
# 97 "Stopped"
# 98 "Not Compiled"
################################################################################
*)
echo "${BinFileDirectory}/dsjob -run -mode RESET -wait $DSPROJNAME $DSJOBNAME"
${BinFileDirectory}/dsjob -run -mode RESET -wait $DSPROJNAME $DSJOBNAME
RETURN_VALUE=$?
if [ $RETURN_VALUE -ne 0 ]
then
echo "Unable to reset job $DSJOBNAME already running..Job run Failed." >> $DSLOG_FILE
exit $RETURN_VALUE
fi
esac
################################################################################
# Execute job here
################################################################################
echo "${BinFileDirectory}/dsjob -run -jobstatus -wait $DSPROJNAME $DSJOBNAME"
${BinFileDirectory}/dsjob -run -jobstatus -wait $DSPROJNAME $DSJOBNAME
RETURN_VALUE=$?
################################################################################
# NOT Runnable Job Status (reset job)
# 0 "Running"
# 3 "Aborted"
# 8 "Failed validation"
# 13 "Failed validation"
# 96 "Aborted"
# 97 "Stopped"
# 98 "Not Compiled"
################################################################################
case $RETURN_VALUE in
1|2)
echo "Job $DSJOBNAME completed successfully "
;;
0)
echo "Error: $DSJOBNAME job failed. error code was -ALREADY RUNNING- Code $RETURN_VALUE" >> $DSLOG_FILE
;;
3|96)
echo "Error: $DSJOBNAME job failed. error code was -ABORT- Code $RETURN_VALUE" >> $DSLOG_FILE
;;
8|13)
echo "Error: $DSJOBNAME job failed. error code was -Failed Validation- Code $RETURN_VALUE" >> $DSLOG_FILE
;;
97)
echo "Error: $DSJOBNAME job failed. error code was -Stopped- Code $RETURN_VALUE" >> $DSLOG_FILE
;;
9)
echo "Error: $DSJOBNAME job failed. error code was -Not Compiled- Code $RETURN_VALUE" >> $DSLOG_FILE
;;
*)
echo "Error: $DSJOBNAME job failed. error code was $RETURN_VALUE" >> $DSLOG_FILE
;;
esac
EXIT_STATUS=$?
if [ ${EXIT_STATUS} -ne 0 ]
then
echo "Error in the Shell for calling the Datastage job sequencer:" $DSJOBNAME >>$DSLOG_FILE
else
echo "Successfully executed the Shell for calling the Datastage job sequencer:" $DSJOBNAME >>$DSLOG_FILE
fi
DT_END=`date`
echo "Execution Ended on $DT_END with Shell Status as :"$EXIT_STATUS>>$DSLOG_FILE
echo "--------------------------------------------">>$DSLOG_FILE
exit ${EXIT_STATUS}
Avishek Mukherjee
Data Integration Architect
Chicago, IL, USA.
kcbland
Participant
Posts: 5208 Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:
Post
by kcbland » Thu Aug 03, 2006 3:42 pm
Instead of hardcoding the path to the engine, change it to:
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle