Processing files in a directory

ray.wurlod · Post by **ray.wurlod** » Fri May 11, 2007 3:34 pm

Both the Server Job Developers Guide and the Parallel Job Developers Guide have a chapter on job sequences, in which all available job sequence components are described. You can also garner information from on-line help in Designer.

sashah · Post by **sashah** » Mon May 14, 2007 11:23 am

To execute the DSRun job comman from prompt do we need to write the script in C? The example in the Server Job Developers guide is written in C

ray.wurlod wrote:Both the Server Job Developers Guide and the Parallel Job Developers Guide have a chapter on job sequences, in which all available job sequence components are described. You can also gar ...

chulett · Post by **chulett** » Mon May 14, 2007 11:45 am

There's no such thing as a 'script' written in C. You're probably seeing example C code from the dsjob application itself or how some of the functions themselves are implemented in it.

There are no script examples in the Guide from what I recall, but descriptions of all of the functions available as part of dsjob. You'd need to build an appropriate script using whichever of those components are applicable for your situation.

If you're not that much of a scripting wiz, there are one or two full blown examples here users have posted of 'wrapper' scripts for dsjob that a search would turn up. Perhaps one of them could help you get started?

sashah · Post by **sashah** » Mon May 14, 2007 12:08 pm

Thank you for all your help.
Can you tell me which reference manual to refer to for calling the DSJob from a Unix prompt. I found some older post in this forum regarding the same but I could not find the reference manual to refer to. I have already written a shell script that loops through files in a directory and would like to pass the file name as a parameter.

Thanks once again.

chulett wrote:There's no such thing as a 'script' written in C. You're probably seeing example C code from the dsjob application itself or how some of the functions themselves are implemented in it.

There are no script examples in the Guide from what I recall, but descriptions of all of the functions available as part of dsjob. You'd need to build an appropriate script using whichever of those components are applicable for your situation.

If you're not that much of a scripting wiz, there are one or two full blown examples here users have posted of 'wrapper' scripts for dsjob that a search would turn up. Perhaps one of them could help you get started?

DSguru2B · Post by **DSguru2B** » Mon May 14, 2007 12:23 pm

Check the section 'Command Line Interface' in Server Job Developer's Guide.

sashah · Post by **sashah** » Mon May 14, 2007 12:24 pm

Thank you.

DSguru2B wrote:Check the section 'Command Line Interface' in Server Job Developer's Guide.

Ed Purcell · Post by **Ed Purcell** » Mon May 14, 2007 12:26 pm

nick.bond wrote:
or you could use wildcards if your file names are amenable)
Ray, I recently found that I couldn't use wildcards in the filter command. Craig also found the same issue. Do you know that it can be done? Is there some specific syntax to get it to work? Or is it just not working in our versions of DS. 7.5.1

Do you mean using wild cards in a File Pattern as the input into a sequential file stage in EE / PX?

chulett · Post by **chulett** » Mon May 14, 2007 12:28 pm

No, wildcards in the Filter option of the Server Sequential File stage.

sashah · Post by **sashah** » Mon May 14, 2007 12:31 pm

I am still not clear on how to use the Filter option to process files in a directory.

chulett wrote:No, wildcards in the Filter option of the Server Sequential File stage. ...

ray.wurlod · Post by **ray.wurlod** » Mon May 14, 2007 4:15 pm

I haven't had the chance to check yet. It may be that you need to quote the entire thing (remember parameters with spaces?).

chulett · Post by **chulett** » Tue May 15, 2007 7:35 am

sashah wrote:I am still not clear on how to use the Filter option to process files in a directory.

By using wildcards there to 'cat' multiple files to standard out, which the Sequential File stage would then read as input. Saves the disk space needed for the combined file if it works for you, it wouldn't for me last time I tried. And tried. I stopped trying.

sashah · Post by **sashah** » Tue May 15, 2007 7:57 am

Thank you for your reply. I am going to try this. However I had a question related to command line interface. How do you pass multiple parameters to DSJob ?

Is it -param name1=value1,name2=value2 or -param name1=value1 -param name2=value2

Thank you again.

chulett wrote:
sashah wrote:I am still not clear on how to use the Filter option to process files in a directory.
By using wildcards there to 'cat' multiple files to standard out, which the Sequential File stage would then read as input. Saves the disk space needed for the combined file if it works for you, it wouldn't for me last time I tried. And tried. I stopped trying. :wink:

chulett · Post by **chulett** » Tue May 15, 2007 8:02 am

sashah wrote:or -param name1=value1 -param name2=value2

That way.

sashah · Post by **sashah** » Tue May 15, 2007 8:04 am

Thank you

chulett wrote:
sashah wrote:or -param name1=value1 -param name2=value2
That way.

sashah · Post by **sashah** » Tue May 15, 2007 8:15 am

I get Invalid Arguments.

This is the command I am executing.

eval $DATASTAGE_HOME/DSEngine/bin/dsjob -run -warn -param accountNumber=$accountNumber -param fileName=$fileName -param lookUpFile=$lookupFilePath -param outputFileName=$outputFile -warn 5 MuniStQa MLConvertibleHistoryLoad

Am I missing something.

Thank you

sashah wrote:Thank you
chulett wrote:
sashah wrote:or -param name1=value1 -param name2=value2
That way.

DSXchange

Processing files in a directory

Executing DSRunJob command