Is it necessary to RESET the job after Run

[Atanu]

Hi ,

I am running DSJOB though shell script.

Is it necessary to run the DSJOB in NORMAL mode first and then RESET mode.

What the actually Usage of RESET mode and what it does?

Thanks,
Vj

[chulett]

RESET mode from the command line does the same thing that resetting the job from the Director does. It is only needed after a job has Aborted or has otherwise gotten itself into a state where the job cannot be run.

Typically, any dsjob shell script would check the job and RESET it before running it, and then only if it needed to be reset. I wouldn't immediately reset an aborted job after it aborts as you lose easy access to the process metadata - the link row counts and rows per second from the aborted run.

[ray.wurlod]

The purpose of RESET is to restore the job to a runnable state. This might involve dropping or resetting database cursors, rolling back pending transactions, closing open text files, clearing locks; anything.

Best practice is as implemented in job sequences "reset if required, then run". There is rarely (never?) any value in resetting a job immediately after a run; as Craig noted, not doing so makes access to diagnostic information about the failure easier to achieve.

[kduke]

Here is a shell script to run a job. It checks to see if you need to reset and then does a reset before running.

Code: Select all

#!/bin/ksh
################################################################################
# File Name    : RunJob.ksh
# Description  : runs DataStage job 
#              : then ftp's file for them to download
# Author       : Kim Duke
# Created      : 06/09/2005
# Modified     : 
# Invocation   : This process is called by autosys
# Prerequisites: None
# 
# Steps        : 1) extract the summary report
#                2) rename the summary report to the date-specific filename
#                3) push the summary report to the ftp site
#                4) push the .done file to the ftp site
#
# Change History
# Who              When        What
# ###############  ##########  #################################################
# Kim Duke         06/09/2005  Original Version
#
################################################################################

THIS_SCRIPT="RunJob.ksh"
EMAIL_USERS="Kim_G_Duke@hotmail.com"

FTP_SERVER="ftp.xxx.com"
FTP_USER="User Password"
FTP_DIR="/export/homeoutgoing"

CURRENT_DATE=`date +%Y%m%d`
EXTRACT_DATE=`date +'%Y%m%dT%H%M%S'`
DSHOME=`cat /.dshome`
PROJECT=$1
shift
DSJOB=$1
shift
# -param ParamName=Value
SET_PARAMS="$*"
cd $DSHOME
. ./dsenv
cd ../Projects/$PROJECT/Staging

################################################################################
# Check job status here
################################################################################
JOB_STATUS=`dsjob -jobinfo $PROJECT $DSJOB | head -1 | cut -d"(" -f2 | cut -d")" -f1`
echo "Before run JOB_STATUS=$JOB_STATUS"
case $JOB_STATUS in
################################################################################
# 0 "Running"
################################################################################
   0)
      echo "Job $DSJOB already running." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS 
      exit 1
      ;;
################################################################################
# Runnable Job Status (do nothing)
# 1 "Finished"
# 2 "Finished (see log)"
# 9 "Has been reset"
# 11 "Validated OK"
# 12 "Validated (see log)"
# 21 "Has been reset"
# 99 "Compiled"
################################################################################
   1|2|7|9|11|12|21|99)
      :
      ;;
################################################################################
# NOT Runnable Job Status (reset job)
# 0 "Running"
# 3 "Aborted"
# 8 "Failed validation"
# 13 "Failed validation"
# 96 "Aborted"
# 97 "Stopped"
# 98 "Not Compiled"
################################################################################
   *)
      echo "dsjob -run -mode RESET -wait $PROJECT $DSJOB"
      dsjob -run -mode RESET -wait $PROJECT $DSJOB
      RETURN_VALUE=$?
      if [ $RETURN_VALUE -ne 0 ] 
      then
         echo "Unable to reset job $DSJOB alreaady running." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS 
         exit $RETURN_VALUE
      fi
esac

################################################################################
# Run job here
################################################################################
echo "dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB"
dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB
RETURN_VALUE=$?
if [ $RETURN_VALUE -eq 1  -o $RETURN_VALUE -eq 2 ]
then
   echo job completed successfully
   EMAIL_MSG=`dsjob -report $PROJECT $DSJOB`
else
   echo "Error: $DSJOB job failed. error code was $RETURN_VALUE"
   echo "Job $DSJOB failed." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS 
   exit $RETURN_VALUE
fi

BASE_FILENAME=${DSJOB}_${EXTRACT_DATE}
FINAL_FILENAME=${BASE_FILENAME}.txt
DONE_FILENAME=${BASE_FILENAME}.done

################################################################################
# cat header onto job
################################################################################
echo "cat ${DSJOB}_header.txt ${DSJOB}.txt >$FINAL_FILENAME"
cat ${DSJOB}_header.txt ${DSJOB}.txt >$FINAL_FILENAME

echo done > $DONE_FILENAME

echo "ftp -n ${FTP_SERVER}"

ftp -n ${FTP_SERVER} << EOF2
user $FTP_USER

prompt
cd ${FTP_DIR}

put ${FINAL_FILENAME}
ls ${FINAL_FILENAME}

put ${DONE_FILENAME}
ls ${DONE_FILENAME}

bye
EOF2

RETURN_VALUE=$?
if [ $RETURN_VALUE -ne 0 ]
then
   echo "Ftp to $FTP_SERVER failed." 
   echo "Ftp to $FTP_SERVER failed." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS 
   exit $RETURN_VALUE
fi

LINES_SENT=`wc -l ${FINAL_FILENAME}`
FINISHED_DATE=`date +'%Y%m%dT%H%M%S'`

mailx -s "${THIS_SCRIPT} finished." $EMAIL_USERS << MEOF

Shell script : ${THIS_SCRIPT}
Sent files   : ${FINAL_FILENAME}
             : ${DONE_FILENAME}
To Server    : ${FTP_SERVER}
To Dir       : ${FTP_DIR}
Lines sent   : ${LINES_SENT}
Started      : ${EXTRACT_DATE}
Finished     : ${FINISHED_DATE}

$EMAIL_MSG

MEOF

[richdhan]

Hi Kim,

It is a good script but I have some clarifications.

################################################################################
# Runnable Job Status (do nothing)
# 1 "Finished"
# 2 "Finished (see log)"
# 9 "Has been reset"
# 11 "Validated OK"
# 12 "Validated (see log)"
# 21 "Has been reset"
# 99 "Compiled"
################################################################################
1|2|7|9|11|12|21|99)

1. What is the job status for 7.

echo "Unable to reset job $DSJOB alreaadyrunning."

2. A spelling mistake which needs to be corrected

3. This question is not a Datastage question. When doing ftp how do I have multiple versions of the same file without over writing the file.

TIA
Rich

[kduke]

I found the spelling error. most of these statuses I figured out by hacking. Some are in DSINCLUDE JOBCONTROL.H. I do not know what status 7 is.

It is hard not to overwrite something in ftp. You would almost need to do 2 calls. One to do a ls and another to put it if it is not there. This script makes a unique file by adding the date and time to the name.

[charles.craig@physiciansm]

################################################################################
# Run job here
################################################################################
echo "dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB"
dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB
RETURN_VALUE=$?
if [ $RETURN_VALUE -eq 1 -o $RETURN_VALUE -eq 2 ]
then
echo job completed successfully
EMAIL_MSG=`dsjob -report $PROJECT $DSJOB`
else
echo "Error: $DSJOB job failed. error code was $RETURN_VALUE"
echo "Job $DSJOB failed." | mailx -s "Error: ${THIS_SCRIPT} failed." $EMAIL_USERS
exit $RETURN_VALUE
fi

Is there value in checking the RETURN_VALUE=$? after the -run command? What does the $? return?

Is it better to reinterrogate the -jobinfo for a status other than 0, 1 or 2? I.E.

Job_Status=`dsjob -jobinfo $DSProject $DSJob | head -1 | cut -d"(" -f2 | cut -d")" -f1`