Wrapped stage

[hhh]

Hello friends,

Have created one shell script which has two parameters.
Have requirement to call this shell script using Wrapped stage.
please share your ideas to design this job,like what are stages required ? etc.

Thanks & Regards
Hman

[ray.wurlod]

Do you want to invoke this script once, or once for every row processed?

[hhh]

Hi,

I want to pass parameters like pathname and retention period, and if i enter retention period, so file which are older than retention period should be removed from particular path.
My requirement is to enter dynamically pathname and retention period.

Thanks & regards
Hman

Do you want to invoke this script once, or once for every row processed?

[ray.wurlod]

You avoided the question. Do you want to invoke this shell script once, or once for every row processed in a job? The answer to your original question depends on your answer to this one.

[hhh]

I want to invoke shell script once

You avoided the question. Do you want to invoke this shell script once, or once for every row processed in a job? The answer to your original question depends on your answer to this one.

[ray.wurlod]

Then all you need is a before-job subroutine executing ExecSH with your script's path and command line arguments (all can be job parameter references) in the Input Values field.

A Wrapped stage would execute the script for every row processed by the job.

[hhh]

Hi Ray,

In job we dont want to use before-job subroutine or after-job subroutine,we need to make generic solution. so how can we achieve
solution without any subroutines ?

Then all you need is a before-job subroutine executing ExecSH with your script's path and command line arguments (all can be job parameter references) in the Input Values field.

A Wrapped stage would execute the script for every row processed by the job.

[roy]

Hi,
You can also wrap the script call in a DS Routine and call it in a before/after routine or as a routine activity in a sequence job.
Having this in a Routine sounds generic to me.

IHTH,

[Ultramundane]

Could you use the External Target Stage. The External Stage will call your script one time and you will need to process standard input in a loop. You can also invoke your script with command line arguments. I typically use awk to process the contents, but you could also use just korn shell and not any other utilities.

In your script, just do a cat - to redirect standard input passed to the shell to whatever command you would like. This command could be awk, while, for, a block, etc. You may want to format your data so that it contains a field seperator. You can set IFS (ksh) or FS (awk) to that value. This can make processing the data much easier.

Below is a really simple example of script which accepts a delimiter and filename for command line arguments. Input is passed in through standard input/input redirection into the program.

Code: Select all

#!/bin/ksh

#################################################################################################
##
## Author: Ryan Putnam
##
#################################################################################################

## SOURCE ENVIRONMENT
. ~/.dsadm

## TRAP FOR EXIT AND ABNORMAL TERMINATION
trap "rm -f ${BTMP}/*.$$ >/dev/null 2>&1;rm -f ${BTMP}/*$$.doc >/dev/null 2>&1" EXIT INT TERM QUIT

###########
## USAGE ##
###########
F_USAGE()
{
   print -u2 "ERROR: ${1}"
   shift

   print -u2 "USAGE: ${1##*/} -D '<Delim>'"
   print -u2 "USAGE: ${1##*/} -F '<File>'"
   print -u2 "USAGE: ${1##*/} -H '<HELP>' -h '<HELP>'"
   exit 10
}

## Lists Usage Information
if [[ "${1}" = "-h" || "${1}" = "-H" ]]; then
   F_USAGE "$0"
fi

## MAP INPUT TO VARIABLES
while getopts D:F: opt
do
   case $opt in
   (D) AFS="$OPTARG" ;;
   (F) FILE="$OPTARG" ;;
   (*) F_USAGE "$0" ;;
   esac
done

####################
## VALIDATE PARMS ##
####################
[[ -z "${AFS}" ]] && F_USAGE "You must specify a delimiter"
[[ -z "${FILE}" ]] && F_USAGE "You must specify a file"

##############
## MAINLINE ##
##############
cat - \
| awk -vAFS="${AFS}" -vFILE="${FILE}" 'BEGIN{FS=AFS;OFS=AFS;}
{
      print $0 >FILE;
}'

exit 0

[ray.wurlod]

You don't need to create a before/after subroutine. Use ExecSH, which is supplied with DataStage. I believe that this is sufficiently generic a solution.

[hhh]

Hi,

Tried to call unix script with external target stage however when i specify pathname to call unix script in destination programme i got warning like
pathname/del.sh not execute and error like export failed.
with pathname i have specified two arguments also

can u please suggest on this issue ?

Thanks
Hman

Could you use the External Target Stage. The External Stage will call your script one time and you will need to process standard input in a loop. You can also invoke your script with command line arguments. I typically use awk to process the contents, but you could also use just korn shell and not any other utilities.

In your script, just do a cat - to redirect standard input passed to the shell to whatever command you would like. This command could be awk, while, for, a block, etc. You may want to format your data so that it contains a field seperator. You can set IFS (ksh) or FS (awk) to that value. This can make processing the data much easier.

Below is a really simple example of script which accepts a delimiter and filename for command line arguments. Input is passed in through standard input/input redirection into the program.

Code: Select all

#!/bin/ksh

#################################################################################################
##
## Author: Ryan Putnam
##
#################################################################################################

## SOURCE ENVIRONMENT
. ~/.dsadm

## TRAP FOR EXIT AND ABNORMAL TERMINATION
trap "rm -f ${BTMP}/*.$$ >/dev/null 2>&1;rm -f ${BTMP}/*$$.doc >/dev/null 2>&1" EXIT INT TERM QUIT

###########
## USAGE ##
###########
F_USAGE()
{
   print -u2 "ERROR: ${1}"
   shift

   print -u2 "USAGE: ${1##*/} -D '<Delim>'"
   print -u2 "USAGE: ${1##*/} -F '<File>'"
   print -u2 "USAGE: ${1##*/} -H '<HELP>' -h '<HELP>'"
   exit 10
}

## Lists Usage Information
if [[ "${1}" = "-h" || "${1}" = "-H" ]]; then
   F_USAGE "$0"
fi

## MAP INPUT TO VARIABLES
while getopts D:F: opt
do
   case $opt in
   (D) AFS="$OPTARG" ;;
   (F) FILE="$OPTARG" ;;
   (*) F_USAGE "$0" ;;
   esac
done

####################
## VALIDATE PARMS ##
####################
[[ -z "${AFS}" ]] && F_USAGE "You must specify a delimiter"
[[ -z "${FILE}" ]] && F_USAGE "You must specify a file"

##############
## MAINLINE ##
##############
cat - \
| awk -vAFS="${AFS}" -vFILE="${FILE}" 'BEGIN{FS=AFS;OFS=AFS;}
{
      print $0 >FILE;
}'

exit 0

[Ultramundane]

Make sure that you mark the script as executable and readable.

I setup the stage as follows.

Input -> Properties -> Target -> Destination Program
/Work/Ascential/script/example_awk.ksh -D"?" -F"/Work/Ascential/tmp/mystuff.txt"

Input -> Properties -> Target -> Target Method
Specific Program(s)

Input -> Format -> Field Defaults -> Delimiter
?
Note: I set to a "?" because I passed -D"?" to the program.

Input -> Format -> Field Defaults -> Quote
none
Note: I set to none to save space and because this property is useless to me.

[hhh]

Hi Ultramundane,
I setup the stage as follows :
Input -> Properties -> Target ->Destination Program
pathname/del.sh pathname retentionperiod
here after pathname/del.sh ,pathname and retentionperiod are two arguments which i want to pass in shell script.
Input -> Properties -> Target -> Target Method
Target method: specific programs
Input->format->field defaults
none.
have setup properties like this,still i am getting same warning and error.

please advise on same

Thanks
Hman

Make sure that you mark the script as executable and readable.

I setup the stage as follows.

Input -> Properties -> Target -> Destination Program
/Work/Ascential/script/example_awk.ksh -D"?" -F"/Work/Ascential/tmp/mystuff.txt"

Input -> Properties -> Target -> Target Method
Specific Program(s)

Input -> Format -> Field Defaults -> Delimiter
?
Note: I set to a "?" because I passed -D"?" to the program.

Input -> Format -> Field Defaults -> Quote
none
Note: I set to none to save space and because this property is useless to me.