Page 1 of 1

Posted: Tue Jul 12, 2011 4:31 pm
by ray.wurlod
1. You don't need to count anything.
2. You can not include a unix command in a Convert() function.
3. Use the ls command to sort the file names. Translate the linefeeds into commas.
4. Have your StartLoop command process a "list of things" comma-delimited list of file names.
5. Within the loop decide which action to take based on file code (filename[4,2]).

Posted: Tue Jul 12, 2011 6:37 pm
by hsahay
Thanks Ray.

Please let me know how to pass command output as a parameter in the parameter set.

following is the error:
______________
job_name.Job_Activity_36.BeforeJob (ExecSH): Error when executing command: head -1 Execute_Command.$CommandOutput>/Directory/HDR_FILE.txt
*** Output from command was: ***
head: cannot open `Execute_Command_27.' for reading: No such file or directory
______________

In my job I need to get the header of the file for processing and to do that I have used unix command.


When I pass file name as is job works fine, but now I want to run the the job for multiple files
one by one and that's the reason I want to pass file name as a parameter which I get from commandoutput in to the parameterset.

What I did is replaced file name parameter in the parameter set with
Execute_Command.$CommandOutput, expecting this will be replaced with file name.

Posted: Wed Jul 13, 2011 5:37 pm
by ray.wurlod
You can not use activity variables as values for parameters, whether they're in parameter sets or not. You need to provide the activity variable reference when setting the value of a parameter, for example within the Job tab of a Job activity or via a -param option on a dsjob command line.

Posted: Thu Jul 14, 2011 6:25 am
by netgurutoo
I am not sure if this is any help but you can change values in Parametersets only in the Values files.
In the Parameterset you can have many value files created in the Values tab of the parameterset. Column one is Value File Name... lets say your value file is OtherParms.txt.... all other columns are you Parameter names you can change this value file in your first job by writing to the file or over writting it. I think the file is in the root but double check and you can view it to see the format for your metadata.. So basically just write your new value/s to OtherParms.txt then kick off a second job and use that value file.

Posted: Thu Jul 21, 2011 2:20 pm
by hsahay
Hi Ray,

The problem is I cannot view inividual parameters of the parameter set with in the job tab of a job activity in v8.5.

If that is poosible, as you said I could pass the activity variable reference when setting the value of a parameter.

Thanks.

Posted: Thu Jul 21, 2011 2:24 pm
by hsahay
netgurutoo,

I didn't understand how to do below said statement.
"you can change this value file in your first job by writing to the file or over writting it."

I didn't quite get the whole thing. Can you please explain more...

Posted: Thu Jul 21, 2011 6:11 pm
by ray.wurlod
You can create one or more values files for the Parameter Set.

Write values into the values file before invoking the job or sequence that consumes the Parameter Set values.

When you run the job, specify one of these file names as the "value" of the Parameter Set, rather than (as pre-defined), in particular the name of the values file that you wrote values into above.

NEW TO DATASTAGE

Posted: Thu Jul 21, 2011 7:39 pm
by VVVRAVINDRA999
:oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops:

Posted: Thu Jul 21, 2011 8:07 pm
by ray.wurlod
The answers have been very specific. The solution is to write parameter name=value pairs into a file and to configure the job to have the Parameter Set read its values from that file.

Please take more care with your written English on DSXchange. Not everyone here has English as a first language, so it's tough enough for them when a professional standard of written English is maintained. SMS-style abbreviations and idiosyncratic spellings only exacerbate this problem.

Posted: Thu Jul 21, 2011 8:42 pm
by hsahay
Ray,

I understood the the concept of value files. Thank you.

But I still can't pass the activity variable reference when setting the value of a parameter .

The reason is I cannot view inividual parameters and their values of the parameter set with in the job tab of a job activity in v8.5.

I believe we can view parameters and their values in the job tab in v7.5

Hope I'm not missing anything.

Thanks again.

Posted: Thu Jul 21, 2011 10:53 pm
by ray.wurlod
You set the value by writing it into the values file.

You set the "value" of the Parameter Set when you submit the job to be run to the name of the values file.

Posted: Fri Jul 22, 2011 7:04 pm
by hsahay
I could create a value file (ValueFile.txt) which looks like

ValueFile.txt
$DB=XYZ
$INPUTFILE=/project/data/data.file.in.1.txt
$OUTPUTFILE=/project/data/data.file.txt

What I would like to do is replace the $INPUTFILE
with
data.file.in.2.txt
data.file.in.3.txt etc...

My job design
Routine Activity1--->RoutineActivity2--->Job that reads ValueFile

Routine Activity1-->ls -lrt data.file.in.* |head -1
Retrieves one file at a time from source directory

Routine Activity2--->echo output of routine1>>ValueFile.txt
takes the output of Routine Activity1(filename) and writes(appends) to the value file as

$DB=XYZ
$INPUTFILE=/project/data/data.file.in.1.txt
$OUTPUTFILE=/project/data/data.file.txt
/project/data/data.file.in.2.txt

which I is not right...it should update the value file like

$DB=XYZ
$INPUTFILE=/project/data/data.file.in.2.txt
$OUTPUTFILE=/project/data/data.file.txt

Let me know where I'm missing and what to do to fix..please.

Thanks.

Posted: Fri Jul 22, 2011 11:20 pm
by chulett
You got exactly what you coded when you simply appended the new value to the current file. In order to "update" the values file, you'd need to read it and then write it out with that specific record replaced with the new value to a new filename and then replace the original file with the new file.

That or get creative with something like the sed command.

Posted: Wed Jul 27, 2011 3:48 pm
by hsahay
I was able to do with the following command

sed -e 's!$FILE=/project/data/data.file.in.1.txt!$FILE=/ABC/data_ABC/data_ABC.file.in.2.txt! file.txt > Newfile.txt'

Thanks.

Posted: Wed Jul 27, 2011 4:25 pm
by chulett
So... resolved?