Passing parameters to a sequence via text file

[pravin1581]

Hi All,

We have a situation where we need to pass the parameter value to a sequence via a text file.The parameters will be read from the text file and passed downsteam to the sequence.
Its important that the parameter should be allocated correctly.The situation is something like this:

Parameter file has the following values:

Code: Select all

    Table Name 1, Column Name 1, Table name 2, Column name 2
    Table Name 3, Column Name 3 Table name 4, Column name 4
   ......

The sequence should run first for the parameters "Table Name 1, Column Name 1, Table name 2, Column name 2" and if it runs again then it should run for the parameters "Table Name 3, Column Name 3, Table name 4, Column name 4"

The parameters need to be passed in 2 jobs downstream.

Thanks.

[shamshad]

Create two files

(1) 1 shell script (example- loadcountry.sh) that will execute your job can be like one below

START SHELL SCRIPT

. /companydir/paramfiles/paramvalues.txt

######################################################################
# DataStage Parameter Processing:
######################################################################
cd $DSHOME
cd `cat /.dshome`
. ./dsenv
bin/dsjob -run -mode NORMAL \
-param 'tddbname'=$ss_tddbname \
-param 'filepath'=$ss_filepath -param 'tdservname'=$ss_tdservname \
-param 'paramfilepath'=$ss_paramfilepath \
-param '$SCRIPT_DIR'=$ss_script_dir $ss_ds_proj $ss_ds_seqjob

END SHELL SCRIPT


(2) Now define all parameters in paramvalues.txt as below

ss_tddbname=yourdb_name
ss_filepath=/companydir/sourcefiles
ss_paramfilepath=/companydir/paramfiles/
ss_script_dir=/companydir/scriptfiles
ss_ds_proj=XXX_DLP_proj

(3) Then, you can execute the shell script as sh loadcountry.sh

Everytime you add more parameters to paramvalues.txt, pass the same parameters to the execute command in the shell script in #1.

[pravin1581]

Create two files

(1) 1 shell script (example- loadcountry.sh) that will execute your job can be like one below

START SHELL SCRIPT

. /companydir/paramfiles/paramvalues.txt

######################################################################
# DataStage Parameter Processing:
######################################################################
cd $DSHOME
cd `cat /.dshome`
. ./dsenv
bin/dsjob -run -mode NORMAL \
-param 'tddbname'=$ss_tddbname \
-param 'filepath'=$ss_filepath -param 'tdservname'=$ss_tdservname \
-param 'paramfilepath'=$ss_paramfilepath \
-param '$SCRIPT_DIR'=$ss_script_dir $ss_ds_proj $ss_ds_seqjob

END SHELL SCRIPT


(2) Now define all parameters in paramvalues.txt as below

ss_tddbname=yourdb_name
ss_filepath=/companydir/sourcefiles
ss_paramfilepath=/companydir/paramfiles/
ss_script_dir=/companydir/scriptfiles
ss_ds_proj=XXX_DLP_proj

(3) Then, you can execute the shell script as sh loadcountry.sh

Everytime you add more parameters to paramvalues.txt, pass the same parameters to the execute command in the shell script in #1.

Thanks for the reply but I have few problems in using the shell script:

1. The dsjob command is not running properly, i need to figure out that first.
2. The sequence should run for the parameters in a way as mentioned earlier, it should run for the parameters defined in the first line, then it should run for the next line until there it reaches the last line.

Thanks.

[jhmckeever]

1. What's the problem with dsjob. Are you getting an error message? You'll find plenty of posts on this topic on dsxchange.

2. So you want to 'parse' a list of parameter values? Presumably there are a variable number of parameter rows, with each row containing a set number of parameters for the specified job. You could try something like ...

Code: Select all

cd $DSHOME
cd `cat /.dshome`
. ./dsenv

cat paramsfile.csv |
while read line
do
   param1 = echo line | cut -f1 -d,
   param2 = echo line | cut -f2 -d,
   param3 = echo line | cut -f3 -d,
   {etc.}

   bin/dsjob -run -mode NORMAL \
   -param 'tddbname'=$param1 \
   -param 'filepath'=$param2 \
   -param 'tdservname'=$param4 \
   {etc.}
done

It's not very elegant, but it should work. Feel free to improve it.

J.

[pravin1581]

1. What's the problem with dsjob. Are you getting an error message? You'll find plenty of posts on this topic on dsxchange.

2. So you want to 'parse' a list of parameter values? Presumably there are a variable number of parameter rows, with each row containing a set number of parameters for the specified job. You could try something like ...

Code: Select all

cd $DSHOME
cd `cat /.dshome`
. ./dsenv

cat paramsfile.csv |
while read line
do
   param1 = echo line | cut -f1 -d,
   param2 = echo line | cut -f2 -d,
   param3 = echo line | cut -f3 -d,
   {etc.}

   bin/dsjob -run -mode NORMAL \
   -param 'tddbname'=$param1 \
   -param 'filepath'=$param2 \
   -param 'tdservname'=$param4 \
   {etc.}
done

It's not very elegant, but it should work. Feel free to improve it.

J.

Thanks for the reply, how the parameters will pass downstream in the sequence.

[ray.wurlod]

Why not just have the job sequence read the file into User Variables and use those downstream? There are a couple of techniques.

One is to read the entire file as a string, parsing out the param=value pairs. This is easily done in a Routine activity. Field() functions can then be used as the derivations of the user variables.

Another is to obtain each param=value pair directly (for example via a routine that invokes the grep command) into the user variable.