Run jobs from command line

kris007 · Post by **kris007** » Thu Mar 16, 2006 8:05 am

Hi All,

I need to run jobs from commmand line in DOS by creating batch files. My DataStage server is remotely installed and I am working on the client machine here. I would like to know, how do I run jobs from command line in this case and schedule these jobs. Do I need to connect to the server remotely and then run the jobs using the dsjob commands? Also, I need some help writing a batch file to pass parameters to this job. How do I go about it. Any insight would be of great help.

Thanks
Kris

kumar_s · Post by **kumar_s** » Thu Mar 16, 2006 8:17 am

Hi,
Search for this topic.
You can get some nice code and informations like this.
Job ran successfully and here is the new and improve script

Code: Select all

#!/bin/ksh 
################################################################################ 
# Description : runs DataStage job 


THIS_SCRIPT="DoSomething.ksh" 
EMAIL_USERS="Michael.Mills@Workcover.nsw.gov.au" 


CURRENT_DATE=`date +%Y%m%d` 
EXTRACT_DATE=`date +'%Y%m%dT%H%M%S'` 
DSHOME="/progs/dstage/Ascential/DataStage/DSEngine" 
DSJOBPATH="/progs/dstage/Ascential/DataStage/DSEngine/bin" 
# PROJECT="PolSubD3" 
PROJECT=$1 
# DSJOB="DoSomethingO2CJobCode" 
DSJOB=$2 
shift 
shift 
PARAMS=$1 
# -param ParamName=Value 
SET_PARAMS="$*" 
cd $DSHOME 
. ./dsenv 
# cd ../Projects/$PROJECT/Staging 
cd $DSJOBPATH 

################################################################################ 
# Check job status here 
################################################################################ 
JOB_STATUS=`dsjob -jobinfo $PROJECT $DSJOB | head -1 | cut -d"(" -f2 | cut -d")" -f1` 
echo "Before run JOB_STATUS=$JOB_STATUS" 
case $JOB_STATUS in 
################################################################################ 
# 0 "Running" 
################################################################################ 
0) 
echo "Job $DSJOB already running." | mailx -s "Error: Job $DSJOB already running ${THIS_SCRIPT} failed." $EMAIL_USERS 
exit 1 
;; 
################################################################################ 
# Runnable Job Status (do something) 
# 1 "Finished" 
# 2 "Finished (see log)" 
# 9 "Has been reset" 
# 11 "Validated OK" 
# 12 "Validated (see log)" 
# 21 "Has been reset" 
# 99 "Compiled" 
################################################################################ 
1|2|7|9|11|12|21|99) 
: 
;; 
################################################################################ 
# NOT Runnable Job Status (reset job) 
# 0 "Running" 
# 3 "Aborted" 
# 8 "Failed validation" 
# 13 "Failed validation" 
# 96 "Aborted" 
# 97 "Stopped" 
# 98 "Not Compiled" 
################################################################################ 
*) 
echo "dsjob -run -mode RESET -wait $PROJECT $DSJOB" 
dsjob -run -mode RESET -wait $PROJECT $DSJOB 
RETURN_VALUE=$? 
if [ $RETURN_VALUE -ne 0 ] 
then 
echo "Unable to reset job $DSJOB alreaady running." | mailx -s "Error: Unable to reset job $DSJOB alreaady running ${THIS_SCRIPT} failed." $EMAIL_USERS 
exit $RETURN_VALUE 
fi 
esac 

################################################################################ 
# Run job here 
################################################################################ 
echo "dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB" 
dsjob -run -jobstatus -wait ${SET_PARAMS} $PROJECT $DSJOB 
RETURN_VALUE=$? 
################################################################################ 
# NOT Runnable Job Status (reset job) 
# 0 "Running" 
# 3 "Aborted" 
# 8 "Failed validation" 
# 13 "Failed validation" 
# 96 "Aborted" 
# 97 "Stopped" 
# 98 "Not Compiled" 
################################################################################ 
case $RETURN_VALUE in 
1|2) 
echo "job completed successfully " 
finished="job completed successfully " 
;; 
0) 
echo "Error: $DSJOB job failed. error code was -ALREADY RUNNING- Code $RETURN_VALUE" 
finished="Error: $DSJOB job failed. error code was -ALREADY RUNNING- Code $RETURN_VALUE" 
;; 
3|96) 
echo "Error: $DSJOB job failed. error code was -ABORT- Code $RETURN_VALUE" 
finished="Error: $DSJOB job failed. error code was -ABORT- Code $RETURN_VALUE" 
;; 
8|13) 
echo "Error: $DSJOB job failed. error code was -Failed Validation- Code $RETURN_VALUE" 
finished="Error: $DSJOB job failed. error code was -Failed Validation- Code $RETURN_VALUE" 
;; 
97) 
echo "Error: $DSJOB job failed. error code was -Stopped- Code $RETURN_VALUE" 
finished="Error: $DSJOB job failed. error code was -Stopped- Code $RETURN_VALUE" 
;; 
9 
echo "Error: $DSJOB job failed. error code was -Not Compiled- Code $RETURN_VALUE" 
finished="Error: $DSJOB job failed. error code was -Not Compiled- Code $RETURN_VALUE" 
;; 
*) 
echo "Error: $DSJOB job failed. error code was $RETURN_VALUE" 
finished="Error: $DSJOB job failed. error code was $RETURN_VALUE" 
;; 
esac 

EMAIL_MSG=`dsjob -report $PROJECT $DSJOB` 
mailx -s "${THIS_SCRIPT} $finished." $EMAIL_USERS << MEOF 
Shell script : ${THIS_SCRIPT} 
$EMAIL_MSG 
MEOF

ray.wurlod · Post by **ray.wurlod** » Thu Mar 16, 2006 8:26 am

Do you think it's likely that they'll have ksh on a Windows platform?

(Actually, if they've got MKS Toolkit, cygwin or similar they may, but that's not the point.)

The techniques in Michael's script are sound, but you need to translate what he's done into BAT file scripting language, which isn't nearly as richly functioned.

Alternatively, why not build most of your control and recoverability into a job sequence, and run that from the command line?

kris007 · Post by **kris007** » Thu Mar 16, 2006 8:32 am

Ray,

I have created a Master Job Sequence which includes other Job sequences in it. All I need to do is run this MasterJob sequence from the command line using the dsjob command.My concern is how do I pass parameters to this Job Sequence. Do I need to write a batch file..If so, It would be great if I could get my hands on a similar batchfile or template so that it would help be working on it. Also, I am not sure if I can schedule and run these jobs( Job Sequence) from my client machine. Need guidance on that.

Thanks
Kris

kumar_s · Post by **kumar_s** » Thu Mar 16, 2006 8:44 am

What sort of parameters you like to pass? If you manage to key in the 'default' in the sequence, it should pass the value to the underlying jobs. So it should be ok to just call the job.
If you need to pass some dynamic parameter which vary for each run, you need to store it ina seperate file and you need to read it and pass it for each run along with dsjob -param option.

kris007 · Post by **kris007** » Thu Mar 16, 2006 8:56 am

Kumar...
I am defining most of my parameters as user-defined environment variables and using them. But there are few parameters which might change for each run. So, to pass those parameters, I am assuming based on what you said, I need to store them in a seperate file. So, my question is do I need to declare those parameters in BASIC.

Kris

kumar_s · Post by **kumar_s** » Thu Mar 16, 2006 9:02 am

All the parameters can be stored in a file and BASIC can be used to just read the file and pass the parameter to the job/sequence as required. And if you are requested(If need to trigger only from command prompt after scheduling), you can call this Batch Job (Which has the BASIC code which calls the jobs) from your DOS Batch file.

maryann · Post by **maryann** » Thu Mar 16, 2006 9:54 am

kumar_s wrote:All the parameters can be stored in a file and BASIC can be used to just read the file and pass the parameter to the job/sequence as required. And if you are requested(If need to trigger only from command prompt after scheduling), you can call this Batch Job (Which has the BASIC code which calls the jobs) from your DOS Batch file.

I am also at the same phase of the project. Have done all the development work ready to go to production .I have worked with MVS edition before and server version is totally different from that of mainframes one. Can any one throw some light on what steps generally needs to be taken from development to production. Can some one put some sample BASIC or DOS CLI which can help me go about writing batch file?

ray.wurlod · Post by **ray.wurlod** » Thu Mar 16, 2006 11:24 am

dsjob -run -param name1=value1 -param name2=value2 -jobstatus project jobname
(No spaces around the "=" characters; that's an artifact of the "italic" tags.)

lstsaur · Post by **lstsaur** » Thu Mar 16, 2006 12:12 pm

Hi Kris,
There is NO WAY that you will be allowed to schedule and run jobs( Job Sequence) from your client machine in the production environment.
My advice to you is that put all the dsjob stuff in the batch file, then add an Execute Command stage, which executes the batch file, to your job sequence. At least, this way your jobs can be scheduled and run by your production environment's scheduling tool.

kris007 · Post by **kris007** » Fri Mar 17, 2006 8:09 am

Alternatively, why not build most of your control and recoverability into a job sequence, and run that from the command line?

Ray,

In this case, how do I go about passing a parameter which is dynamic. Now, I can define all my parameters as environment variables. I have only one parameter which changes during the runtime depending on the action I need to take. So, I would like to know on how to pass this single parameter during runtime. Do I have to go into administrator everytime and change the default value. What I would prefer to do is change that parameter without going into the tool..like may be from the command line. In Help, they have suggested to changed the environment variable during run time by going into the Job Properties page and overriding the default value. Is there an other way we can go about this. As you have suggested, can I run use the

Code: Select all

[b]dsjob -run -param [i]name1=value1[/i][/b] -jobstatus projectname jobname

to pass only this dynamic parameter and use the rest of the parameters as default values.

Any insight would be of great help.

Thanks
Kris

ray.wurlod · Post by **ray.wurlod** » Fri Mar 17, 2006 9:52 am

All of what you've stated is cool, plus there exists the ability to specify job parameters as environment variables and set the default value there to $ENV, which means "pick up the value from the user's environment". Beware, though, that you do not run DataStage jobs; you issue run request and the job is run as a process on the server. Depending on how you've set things up it may or may not run under your particular user ID.

kris007 · Post by **kris007** » Fri Mar 17, 2006 10:22 am

Do I set the default value for parameter as $ENV OR $PROJDEF

And could you please elaborate on

Depending on how you've set things up it may or may not run under your particular user ID.

. Would it make a difference if I set the default value as $PROJDEF.

Thanks
Kris.

ray.wurlod · Post by **ray.wurlod** » Fri Mar 17, 2006 10:28 am

Click on Help within the job parameters dialog to learn the difference between $ENV, $PROJDEF and $UNSET (or just search for them in Help). Yes, it will make a difference.

As to the other question, on Windows you can create a different user ID for running scheduled jobs. It's to that capability that I referred. You would need to ensure that the environment variables for that user were properly set up if running the job from the scheduler.

mystuff · Post by **mystuff** » Thu Jan 03, 2008 3:12 pm

Why does some of the returned codes means the same in the script, like

8,13 and 3,96