DSExecute in a Sequencer is not executing any command

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

DSExecute in a Sequencer is not executing any command

Post by bigpoppa »

One of my Job sequences has only one stage: an "execute command" stage that calls a script called #pPathNameScrpt#/#pSed#, where both params are defined in the job properties box. This command never executes when I run the sequencer, though the run log shows the correct command being called, and a reply of 0. However, if I write out the entire path and script name w/o using any parameters, the correct command is actually executed on the server. Eventually, I will add more stages to the sequencer, but I'd like to get past this hurdle first.

I am wondering why I can't use the parameterized version of the execute command stage. The code below is the DS generated code for the sequencer, and I was hoping someone could review it. Isn't it fishy that the parameter values are never set?

- BP


* Tokens were replaced below as follows:
* Id59%%Result1%%1 <= SED_STRIP_HEADER_TRAILER.$ReturnValue
*
*** [Generated at 2003-12-02 15:48:07]
$INCLUDE DSINCLUDE DSJ_XFUNCS.H
****************************************
* Graphical Sequencer generated code for Job CopyOfSeqOrgProfAndClient
****************************************

*** Sequence start point
GoSub L$59$START
GoTo L$FINISH
**************************************************
L$59$START:
*** Activity "SED_STRIP_HEADER_TRAILER": Execute command
p$59$1 = ""
x$cmd = "#pPathNameScrpt#/#pSed#":" ":p$59$1
x$type = (If System(91) = 1 Then "NT" Else "UNIX")
Call DSExecute(x$type, x$cmd, x$output, x$59)
Call DSLogInfo(DSMakeMsg("DSTAGE_JSG_M_0007\Executed: %1<L>Reply=%2<L>Output from command ====><L>%3", x$cmd:@FM:x$59:@FM:Lower(x$output)), "@SED_STRIP_HEADER_TRAILER")

Id59%%Result1%%1 = x$59
Return
**************************************************
L$FINISH:
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Re: DSExecute in a Sequencer is not executing any command

Post by Teej »

Try it without the #'s.

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

DSExecute in a Sequencer is not executing any command

Post by bigpoppa »

Tried it.. didn't work. Any other ideas?

-BP
mhester
Participant
Posts: 622
Joined: Tue Mar 04, 2003 5:26 am
Location: Phoenix, AZ
Contact:

Post by mhester »

This stage does not work with parameters. It is documented with Ascential and an eCase has been opened. No work around as of yet. I believe they said it would be addressed in 7x.

The parms are not being evaluated and replaced with the value.

Regards,

Michael Hester
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

DSExecute in a Sequencer is not executing any command

Post by bigpoppa »

Nuts! But, thanks for the info.

I actually have several sequencers with similar "execute command" stages that do work, so I guess it's a hit or miss thing.. I wonder why sometimes the stage works and sometimes it doesn't.

- BP
Last edited by bigpoppa on Tue Dec 02, 2003 3:49 pm, edited 1 time in total.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Parameters in the command path are not supported since version 5, in case anyone is wondering.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
mhester
Participant
Posts: 622
Joined: Tue Mar 04, 2003 5:26 am
Location: Phoenix, AZ
Contact:

Post by mhester »

I can't imagine that it has worked if you were passing parameters since this is a known issue since, as Ken states, version 5.

Regards,

Michael Hester
vzmz
Participant
Posts: 36
Joined: Sun Nov 23, 2003 12:10 pm
Location: Dallas

I do have a similar problem

Post by vzmz »

I created a job sequencer in which i have 2 jobs that has to be executed .
But its run only the first job. In the log it say 2nd job executed but i don't see the result. Which indicate that the 2nd job wan't executed.
This is with 7.x on unix
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

DSExecute in a Sequencer is not executing any command

Post by bigpoppa »

Thanks for all the input.

Below is an example of generated sequence code in which there are 2 execute command stages (at L$5$Start, L$6$Start) and one JobActivity(at L$4$Start).

In this example, both L$5 and L$6 use params to generate commands. L$5 always works (i.e. the before_count log gets created with the correct info), L$6 never works. I'm thinking that if I could figure out why L$5 works, I might be able to come up with a hack to make L$6 work. Any ideas?

-BP
-----------------------


* Tokens were replaced below as follows:
* Id6%%Result1%%1 <= Execute_Command_18.$ReturnValue
* Id5%%Result1%%2 <= BEFORE_REC_STATUS.$ReturnValue
* Id4%%Result1%%3 <= BEFORE_RECORD_COUNT.$JobStatus
* Id4%%Result2%%4 <= BEFORE_RECORD_COUNT.$UserStatus
*
*** [Generated at 2003-12-03 09:48:05]
$INCLUDE DSINCLUDE DSJ_XFUNCS.H
****************************************
* Graphical Sequencer generated code for Job CopyOfSeqOrgProfAndClient
****************************************

handle$list = ""
id$list = ""
*** Sequence start point
GoSub L$4$START
GoTo L$WAITFORJOB
**************************************************
L$5$START:
*** Activity "BEFORE_REC_STATUS": Execute command
p$5$1 = ""
x$cmd = "#pPathNameScrpt#/#pRecordCount# #pPathNameLog#/#pInputFile#_beforecount.log":" ":p$5$1
x$type = (If System(91) = 1 Then "NT" Else "UNIX")
Call DSExecute(x$type, x$cmd, x$output, x$5)
Call DSLogInfo(DSMakeMsg("DSTAGE_JSG_M_0007\Executed: %1<L>Reply=%2<L>Output from command ====><L>%3", x$cmd:@FM:x$5:@FM:Lower(x$output)), "@BEFORE_REC_STATUS")

Id5%%Result1%%2 = x$5
If (Id5%%Result1%%2 = 0) Then GoSub L$6$START
Return
**************************************************
L$6$START:
*** Activity "Execute_Command_18": Execute command
p$6$1 = ""
x$cmd = "/usr/bin/sed '1d$d' #pPathNameData#/#pInputFile >> #pPathNameData#/#pInputFile#.dat":" ":p$6$1
x$type = (If System(91) = 1 Then "NT" Else "UNIX")
Call DSExecute(x$type, x$cmd, x$output, x$6)
Call DSLogInfo(DSMakeMsg("DSTAGE_JSG_M_0007\Executed: %1<L>Reply=%2<L>Output from command ====><L>%3", x$cmd:@FM:x$6:@FM:Lower(x$output)), "@Execute_Command_18")

Id6%%Result1%%1 = x$6
Return
**************************************************
L$4$START:
*** Activity "BEFORE_RECORD_COUNT": Initialize job
jb$4 = "SvrTemplateBeforeRecCount":'.':("BEFORE_RECORD_COUNT")
h$4 = DSAttachJob(jb$4, DSJ.ERRNONE)
If (Not(h$4)) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0001\Error calling DSAttachJob(%1)<L>%2", jb$4:@FM:DSGetLastErrorMsg())
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
p$4$1 = (pPassWord)
err$code = DSSetParam(h$4, "pPassWord", p$4$1)
If (err$code <> DSJE.NOERROR) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0002\Error calling DSSetParam(%1), code=%2[E]", "pPassWord":@FM:err$code)
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
p$4$2 = (pUserName)
err$code = DSSetParam(h$4, "pUserName", p$4$2)
If (err$code <> DSJE.NOERROR) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0002\Error calling DSSetParam(%1), code=%2[E]", "pUserName":@FM:err$code)
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
p$4$3 = (pServerName)
err$code = DSSetParam(h$4, "pServerName", p$4$3)
If (err$code <> DSJE.NOERROR) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0002\Error calling DSSetParam(%1), code=%2[E]", "pServerName":@FM:err$code)
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
p$4$4 = "myfile"
err$code = DSSetParam(h$4, "pInputFile", p$4$4)
If (err$code <> DSJE.NOERROR) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0002\Error calling DSSetParam(%1), code=%2[E]", "pInputFile":@FM:err$code)
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
p$4$5 = (pPathNameLog)
err$code = DSSetParam(h$4, "pPathNameLog", p$4$5)
If (err$code <> DSJE.NOERROR) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0002\Error calling DSSetParam(%1), code=%2[E]", "pPathNameLog":@FM:err$code)
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
p$4$6 = (pPathNameScrpt)
err$code = DSSetParam(h$4, "pPathNameScrpt", p$4$6)
If (err$code <> DSJE.NOERROR) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0002\Error calling DSSetParam(%1), code=%2[E]", "pPathNameScrpt":@FM:err$code)
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
p$4$7 = (pPathNameData)
err$code = DSSetParam(h$4, "pPathNameData", p$4$7)
If (err$code <> DSJE.NOERROR) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0002\Error calling DSSetParam(%1), code=%2[E]", "pPathNameData":@FM:err$code)
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
err$code = DSRunJob(h$4, DSJ.RUNNORMAL)
If (err$code <> DSJE.NOERROR) Then
msg$ = DSMakeMsg("DSTAGE_JSG_M_0003\Error calling DSRunJob(%1), code=%2[E]", jb$4:@FM:err$code)
msg$id = "@BEFORE_RECORD_COUNT"; GoTo L$ERROR
End
handle$list<-1> = h$4
id$list<-1> = "4"
Return
**************************************************
L$4$FINISHED:
Id4%%Result1%%3 = DSGetJobInfo(h$4, DSJ.JOBSTATUS)
Id4%%Result2%%4 = DSGetJobInfo(h$4, DSJ.USERSTATUS)
rpt$4 = DSMakeJobReport(h$4, 1, "CRLF")
job$4$status = DSGetJobInfo(h$4, DSJ.JOBSTATUS)
dummy$ = DSDetachJob(h$4)
If (job$4$status = DSJS.RUNOK) Then GoSub L$5$START
GoTo L$WAITFORJOB
**************************************************
L$WAITFORJOB:
If handle$list = "" Then GoTo L$FINISH
handle$ = DSWaitForJob(handle$list)
If handle$ = 0 Then handle$ = handle$list<1>
Locate handle$ In handle$list Setting index$ Then
id$ = id$list<index$>
Del id$list<index$>; Del handle$list<index$>
Begin Case
Case id$ = "4"
GoTo L$4$FINISHED
End Case
End
* Error if fall though
msg$ = DSMakeMsg("DSTAGE_JSG_M_0008\Error calling DSWaitForJob(), code=%1[E]", handle$)
msg$id = "@Coordinator"; GoTo L$ERROR
**************************************************
L$ERROR:
Call DSLogWarn(DSMakeMsg("DSTAGE_JSG_M_0009\Controller problem: %1", msg$), msg$id)
Call DSLogFatal(DSMakeMsg("DSTAGE_JSG_M_0013\Sequence job will abort due to previous unrecoverable errors", ""), msg$id)

**************************************************
L$FINISH:
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Methinks you have a quotes issue:

Code: Select all

x$cmd = "/usr/bin/sed '1d$d' #pPathNameData#/#pInputFile >> #pPathNameData#/#pInputFile#.dat":" ":p$6$1 

For grins put this into a shell script and execute the shell script. It probably works. Pass the shell script your parameter values on the command line.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

Post by bigpoppa »

Ken,

Thanks for your help.

I do have a '#' issue on the #pInputFile param, but I fixed that and it still doesn't work.

I have tried doing:

sed '1d;$d' path/file >> path/newfile

and it works file..

Which set of quotes do you think is the problem? The set encasing the entire statement (it was generated by DS) or the single quote set around the sed option?

- BP
Last edited by bigpoppa on Wed Dec 03, 2003 10:35 am, edited 1 time in total.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

I have been burned in the past using embedded quotes within DSExecute, which is all a sequencer uses in its command stages. I try never to directly execute Unix commands unless they are basically very simple (mv, cat, rm, etc). You have little control over error handling in the command stages.

I prefer to have a shell script control the exceptional handling of whatever it is doing, then return the mindless pass/fail error value.

In your case, I'd put the sed statement into a shell script, complete with redirection nested in the script. This way, all DS has to do is run a script and give it command line parameters. Ex:

bigpoppased.ksh

Code: Select all

sed '1d;$d' $1 >> $2
Put in your command stage:
#pathtoscripts#/bigpoppased.ksh as the command
path/file path/newfile as the parameters

You should see that the command doesn't find the path to scripts, unless your search path magically finds it.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Post Reply