Hi ,
I am running DSJOB though shell script.
Is it necessary to run the DSJOB in NORMAL mode first and then RESET mode.
What the actually Usage of RESET mode and what it does?
Thanks,
Vj
Is it necessary to RESET the job after Run
Moderators: chulett, rschirm, roy
RESET mode from the command line does the same thing that resetting the job from the Director does. It is only needed after a job has Aborted or has otherwise gotten itself into a state where the job cannot be run.
Typically, any dsjob shell script would check the job and RESET it before running it, and then only if it needed to be reset. I wouldn't immediately reset an aborted job after it aborts as you lose easy access to the process metadata - the link row counts and rows per second from the aborted run.
Typically, any dsjob shell script would check the job and RESET it before running it, and then only if it needed to be reset. I wouldn't immediately reset an aborted job after it aborts as you lose easy access to the process metadata - the link row counts and rows per second from the aborted run.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The purpose of RESET is to restore the job to a runnable state. This might involve dropping or resetting database cursors, rolling back pending transactions, closing open text files, clearing locks; anything.
Best practice is as implemented in job sequences "reset if required, then run". There is rarely (never?) any value in resetting a job immediately after a run; as Craig noted, not doing so makes access to diagnostic information about the failure easier to achieve.
Best practice is as implemented in job sequences "reset if required, then run". There is rarely (never?) any value in resetting a job immediately after a run; as Craig noted, not doing so makes access to diagnostic information about the failure easier to achieve.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Here is a shell script to run a job. It checks to see if you need to reset and then does a reset before running.
Code: Select all
#!/bin/ksh
################################################################################
# File Name : RunJob.ksh
# Description : runs DataStage job
# : then ftp's file for them to download
# Author : Kim Duke
# Created : 06/09/2005
# Modified :
# Invocation : This process is called by autosys
# Prerequisites: None
#
# Steps : 1) extract the summary report
# 2) rename the summary report to the date-specific filename
# 3) push the summary report to the ftp site
# 4) push the .done file to the ftp site
#
# Change History
# Who When What
# ############### ########## #################################################
# Kim Duke 06/09/2005 Original Version
#
################################################################################
THIS_SCRIPT="RunJob.ksh"
EMAIL_USERS="Kim_G_Duke@hotmail.com"
FTP_SERVER="ftp.xxx.com"
FTP_USER="User Password"
FTP_DIR="/export/homeoutgoing"
CURRENT_DATE=`date +%Y%m%d`
EXTRACT_DATE=`date +'%Y%m%dT%H%M%S'`
DSHOME=`cat /.dshome`
PROJECT=$1
shift
DSJOB=$1
shift
# -param ParamName=Value
SET_PARAMS="$*"
cd $DSHOME
. ./dsenv
cd ../Projects/$PROJECT/Staging
################################################################################
# Check job status here
################################################################################
JOB_STATUS=`dsjob -jobinfo $PROJECT $DSJOB | head -1 | cut -d"(" -f2 | cut -d")" -f1`
echo "Before run JOB_STATUS=$JOB_STATUS"
case $JOB_STATUS in
################################################################################
# 0 "Running"
################################################################################
0)
echo "Job $DSJOB already running." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS
exit 1
;;
################################################################################
# Runnable Job Status (do nothing)
# 1 "Finished"
# 2 "Finished (see log)"
# 9 "Has been reset"
# 11 "Validated OK"
# 12 "Validated (see log)"
# 21 "Has been reset"
# 99 "Compiled"
################################################################################
1|2|7|9|11|12|21|99)
:
;;
################################################################################
# NOT Runnable Job Status (reset job)
# 0 "Running"
# 3 "Aborted"
# 8 "Failed validation"
# 13 "Failed validation"
# 96 "Aborted"
# 97 "Stopped"
# 98 "Not Compiled"
################################################################################
*)
echo "dsjob -run -mode RESET -wait $PROJECT $DSJOB"
dsjob -run -mode RESET -wait $PROJECT $DSJOB
RETURN_VALUE=$?
if [ $RETURN_VALUE -ne 0 ]
then
echo "Unable to reset job $DSJOB alreaady running." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS
exit $RETURN_VALUE
fi
esac
################################################################################
# Run job here
################################################################################
echo "dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB"
dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB
RETURN_VALUE=$?
if [ $RETURN_VALUE -eq 1 -o $RETURN_VALUE -eq 2 ]
then
echo job completed successfully
EMAIL_MSG=`dsjob -report $PROJECT $DSJOB`
else
echo "Error: $DSJOB job failed. error code was $RETURN_VALUE"
echo "Job $DSJOB failed." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS
exit $RETURN_VALUE
fi
BASE_FILENAME=${DSJOB}_${EXTRACT_DATE}
FINAL_FILENAME=${BASE_FILENAME}.txt
DONE_FILENAME=${BASE_FILENAME}.done
################################################################################
# cat header onto job
################################################################################
echo "cat ${DSJOB}_header.txt ${DSJOB}.txt >$FINAL_FILENAME"
cat ${DSJOB}_header.txt ${DSJOB}.txt >$FINAL_FILENAME
echo done > $DONE_FILENAME
echo "ftp -n ${FTP_SERVER}"
ftp -n ${FTP_SERVER} << EOF2
user $FTP_USER
prompt
cd ${FTP_DIR}
put ${FINAL_FILENAME}
ls ${FINAL_FILENAME}
put ${DONE_FILENAME}
ls ${DONE_FILENAME}
bye
EOF2
RETURN_VALUE=$?
if [ $RETURN_VALUE -ne 0 ]
then
echo "Ftp to $FTP_SERVER failed."
echo "Ftp to $FTP_SERVER failed." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS
exit $RETURN_VALUE
fi
LINES_SENT=`wc -l ${FINAL_FILENAME}`
FINISHED_DATE=`date +'%Y%m%dT%H%M%S'`
mailx -s "${THIS_SCRIPT} finished." $EMAIL_USERS << MEOF
Shell script : ${THIS_SCRIPT}
Sent files : ${FINAL_FILENAME}
: ${DONE_FILENAME}
To Server : ${FTP_SERVER}
To Dir : ${FTP_DIR}
Lines sent : ${LINES_SENT}
Started : ${EXTRACT_DATE}
Finished : ${FINISHED_DATE}
$EMAIL_MSG
MEOF
Mamu Kim
Hi Kim,
It is a good script but I have some clarifications.
3. This question is not a Datastage question. When doing ftp how do I have multiple versions of the same file without over writing the file.
TIA
Rich
It is a good script but I have some clarifications.
1. What is the job status for 7.Kim Duke wrote:################################################################################
# Runnable Job Status (do nothing)
# 1 "Finished"
# 2 "Finished (see log)"
# 9 "Has been reset"
# 11 "Validated OK"
# 12 "Validated (see log)"
# 21 "Has been reset"
# 99 "Compiled"
################################################################################
1|2|7|9|11|12|21|99)
2. A spelling mistake which needs to be correctedKim Duke wrote:echo "Unable to reset job $DSJOB alreaadyrunning."
3. This question is not a Datastage question. When doing ftp how do I have multiple versions of the same file without over writing the file.
TIA
Rich
I found the spelling error. most of these statuses I figured out by hacking. Some are in DSINCLUDE JOBCONTROL.H. I do not know what status 7 is.
It is hard not to overwrite something in ftp. You would almost need to do 2 calls. One to do a ls and another to put it if it is not there. This script makes a unique file by adding the date and time to the name.
It is hard not to overwrite something in ftp. You would almost need to do 2 calls. One to do a ls and another to put it if it is not there. This script makes a unique file by adding the date and time to the name.
Mamu Kim
-
- Participant
- Posts: 15
- Joined: Wed May 11, 2005 12:09 pm
Clarification question
Is there value in checking the RETURN_VALUE=$? after the -run command? What does the $? return?kduke wrote:################################################################################
# Run job here
################################################################################
echo "dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB"
dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB
RETURN_VALUE=$?
if [ $RETURN_VALUE -eq 1 -o $RETURN_VALUE -eq 2 ]
then
echo job completed successfully
EMAIL_MSG=`dsjob -report $PROJECT $DSJOB`
else
echo "Error: $DSJOB job failed. error code was $RETURN_VALUE"
echo "Job $DSJOB failed." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS
exit $RETURN_VALUE
fi
Is it better to reinterrogate the -jobinfo for a status other than 0, 1 or 2? I.E.
Job_Status=`dsjob -jobinfo $DSProject $DSJob | head -1 | cut -d"(" -f2 | cut -d")" -f1`