Executing unix script via "Execute Command" in seq

jjrbikes · Post by **jjrbikes** » Wed Jun 16, 2004 2:30 pm

I have a very simple unix script that runs perfectly when executed natively from the command line. However, when the script is executed via the "Execute Command" stage in a DataStage sequencer it fails miserably. This script simply checks the value of two arguments and, depending on the value of each argument, it may or may not copy one file to another. When executed via DataStage it NEVER copies these files - even when the logic says it should.

I've tried everything I can think of and everything two of my favorite Unix gurus can think of - to no avail. What we've concluded is that it appears the script may not be running on the korn shell....even though my command "forces" it to the korn shell:
ksh $ArchivePath/$kshBinPath/scriptname.ksh "#arg1#" "#arg2#"

for what it's worth, I've changed the command line arguments to be quoted as you see them there, quoted with single quotes and not quoted at all.

Any ideas? Any thoughts? here's the script in it's entirety:

#!/usr/bin/ksh
#
# History:
# 20040611 JJR created.
#

HOME_DIR="/proj/dsadm/Ascential/DataStage/Projects"
PROJ_NAME="proj/user"
CATEGORY="projcategory"

CB=$1
PT=$2

echo $CB
echo $PT

if [ $CB = "PASS" ]
then
cp ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameA1> ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameA2>
fi

if [ $PT = "PASS" ]
then
cp ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameB1> ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameB2>
fi

thanks!!

Jennifer

ogmios · Post by **ogmios** » Wed Jun 16, 2004 2:42 pm

How do you know the script actually executes? Built in some simple echo's to a fixed trace file and see.

Running scripts from DataStage is very similar to running scripts from cron. I would first include the full path to ksh in your execution although using the extra ksh should normally not be required.

What you can also do is to try running the script from within a small BASIC program using DSExecute, and then printing the screenoutput to your log.

Ogmios

chulett · Post by **chulett** » Wed Jun 16, 2004 2:47 pm

As stupid as it sounds - you can not pass Job Parameters to a script using the Execute Command stage. It will look like it's working in the log, but they do not get translated in the actual script execution.

Painful bug been there since Day One with the stage.

chucksmith · Post by **chucksmith** » Wed Jun 16, 2004 3:51 pm

Although the "execute command" stage does not like parameters, you can easily replace it with a "routine activity" stage, and hide your command within it. Use DSExecute to run your command from the routine.

Chuck

chulett · Post by **chulett** » Wed Jun 16, 2004 4:06 pm

Sorry, should have mentioned that as it is the official 'workaround'.

kduke · Post by **kduke** » Wed Jun 16, 2004 6:25 pm

Put an echo in front of the whole statement and >debug.txt to see what is exactly getting executed.

chucksmith · Post by **chucksmith** » Wed Jun 16, 2004 6:32 pm

If you look at the code generated by the seqencer, you will see that it does not recognize parameters in the "Execte Command" stage. Try plan B before spending any more time with this stage, if you need parameters.

Chuck

jjrbikes · Post by **jjrbikes** » Thu Jun 17, 2004 5:27 am

Wow - thanks for all the input. I'm not convinced though that the parameters aren't being recognized -or translated- in the execution of the script since the echo statements I already have in there echo back the correct values of the parameters I'm sending it. Does that make sense? Anyway - since I've already wasted way too much time on this I'll give the "routine activity" stage a shot. Have never worked with that stage...this should be fun!!

Thanks again everyone and have aGREAT day!

Jennifer

chulett · Post by **chulett** » Thu Jun 17, 2004 6:30 am

jjrbikes wrote:Wow - thanks for all the input. I'm not convinced though that the parameters aren't being recognized -or translated- in the execution of the script since the echo statements I already have in there echo back the correct values of the parameters I'm sending it.

Really? $CB and $PT echo out correctly? What exact version of DataStage are you running? If it really is passing parameters then something else must be going on and you may not have any more luck with the routine.

It doesn't look like you have the classic "doesn't work under DataStage" problem - assumptions about your environment. Just because a script runs from the command line doesn't mean it will run under DataStage, all it means is it is syntactically correct. The environment a job runs under is vastly different than that same user's environment at the command line. Write a job that simply issues a 'SET' command, capture the output and you'll see.

As I said, you seem to be providing everything needed in the script and not making any assumptions about what environment variables will be available or where you are when this runs. This is good. I'm also not sure that this even requires the Korn shell, so I don't think your 'not running under Korn' is worth pursuing. Some things to try, if you haven't already...

Try using "ksh -x" and capturing the output as Kim suggested to see what is going on.

Add the curly braces you've used almost everywhere else to the CB and PT variables in your if statement. Shouldn't make a difference, but at this point ya never know.

Good Luck!

kduke · Post by **kduke** » Thu Jun 17, 2004 8:43 am

When you post your code use the code button on the web page before and after. That way we will see everything because if it looks like a html tag then it does not display properly.

jjrbikes · Post by **jjrbikes** » Fri Jun 18, 2004 7:21 am

Kim - thanks for the tip on the "code" button - I'm still new to all this and can use all the help I can get.

"chulett" - I'm running on 7.0.1 on Tru64 (digital unix)
I did attempt to write a routine to accomplish what I wanted this script to do. Notice I said "attempt" - the routine compiles fine but when I "test" it I get this error message: [color=red]Error calling subroutine: DSR_EDIT (Action=2); check DataStage is set up correctly in project <projectname>
(Subroutine failed to complete successfully (30107))[/color]

Suggestions??? I checked - and DSR_EDIT is there....

Now - back to your suggestion to run the DS job with ksh -x to see what was really happening... DO you see what I see???

bRxPharmCareLoadCompareBld2..JobControl (@MoveFailedFileksh): Executed: ksh -x $ArchivePath/$kshBinPath/pharmacareloadcompare.ksh "PASS" "FAIL"
Reply=0
Output from command ====>
+ HOME_DIR=/proj/dsadm/Ascential/DataStage/Projects
+ PROJ_NAME=adwdvl/adw
+ CATEGORY=rxpharmacare
+ CB=PASS
+ PT=FAIL
+ echo PASS
PASS
+ echo FAIL
FAIL
+ [ PASS = PASS ]
+ [ FAIL = PASS ]

Datastage is INDEED receiving the parameters I'm passing - but it is NOT recognizing the "then" statement following a "true" IF statement. It's the "IF-THEN-ELSE" - NOT the parameters that are the problem. Any suggestions for that???

ps - you're correct in assuming the curly braces don't make a difference - I tried that early on.

Thanks Again Everybody!
Jennifer

chucksmith · Post by **chucksmith** » Fri Jun 18, 2004 8:53 am

Why not move all of the logic of building the command into the routine, and then just use the Call DSExecute to execute the command, instead of the whole script?

Chuck

gherbert · Post by **gherbert** » Fri Jun 18, 2004 9:00 am

jjrbikes wrote: if [ $CB = "PASS" ]
then
cp ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameA1> ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameA2>
fi

if [ $PT = "PASS" ]
then
cp ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameB1> ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameB2>
fi

Try enclosing the $CB and $PT within quotes, i.e.

if [ "$CB" = "PASS" ] and if [ "$PT" = "PASS" ]

My guess is that somehow there is leading/trailing whitespace, such that you aren't really getting "PASS" or "FAIL", but "<space>PASS", etc. The debug of the script won't show that as we humans can't "see" whitespace all the time

Hope this helps.

chulett · Post by **chulett** » Fri Jun 18, 2004 9:01 am

jjrbikes wrote:Datastage is INDEED receiving the parameters I'm passing - but it is NOT recognizing the "then" statement following a "true" IF statement. It's the "IF-THEN-ELSE" - NOT the parameters that are the problem. Any suggestions for that???

That's cool.

Must be an O/S thing 'cuz it don't for me on HP/UX the last time I checked. I miss Tru64.

Edit - just tried again and they still do not translate for me - 7.0.1 on HP/UX 11i.

Try this syntax, it has always worked for me:

Code: Select all

if [ "$CB" = "PASS" ] ;then
  cp ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameA1> ${HOME_DIR}/${PROJ_NAME}/${CATEGORY}/seq/<filenameA2> 
fi

Only other comment is I don't see any ELSE logic in your code, are you speaking... metaphorically?

jjrbikes · Post by **jjrbikes** » Fri Jun 18, 2004 12:19 pm

I guess I'm not really tracking with the Call DSExecute thing.

"chulett" -
negative on the [if [ "$CB" = "PASS" ] ;then] code change. I tried that and a number of variations with no success. I then attempted to replace the if-then-fi with a case statement - same results >> and not the ones I'm looking for. There's got to be an answer...perhaps I'll contact Ascential and see what they have to say.

Interesting find that some O/Ss don't recognize the parameters and others do....then again, it apparently makes no difference that Tru64 DOES recognize them.

Thanks Again
Jennifer

DSXchange

Executing unix script via "Execute Command" in seq

Executing unix script via "Execute Command" in seq

Re: Executing unix script via "Execute Command" in

Re: Executing unix script via "Execute Command" in