Dynamically launch job(s) based on list of values

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
wwalker
Premium Member
Premium Member
Posts: 40
Joined: Thu Mar 30, 2006 6:30 am
Location: Near Geneva, Switzerland
Contact:

Dynamically launch job(s) based on list of values

Post by wwalker »

OK...I will try to be concise...

I need to launch a series of jobs (currently 1 job designed as multi-instance) that creates a one target table each instance, based on specific criteria.

One criteria is that one table must be created by company SITE, that will be found in a source file. This also means that one instance of a job must be launched by site. Apparently, it is difficult to identify all possible sites (which seems a little strange...), so I must create a sequence that will dynamically launch one or more instances of the job based on a list of sites built from the source file (one site per job).

I am trying to think about how to best do this...

1. I have thought about building a routine builds an array(comma separated) containing the list of distinct sites built from a given version of the source file, and passing the return value of the routine to a LOOP sequence (which I have never used)...

The problem with the above, is that I also must pass the value of the site that is currently in loop as a parameter for the creation of the table (i.e. table name)...not sure how to do this...

Another way (ideas/suggestions please??) would be to somehow pass the list of distinct sites to a nested condition sequence stage, then for each output link, provide a single value for site...but not sure how to do this either..

...ANY AND ALL ideas are welcome.....!!

thanks!
Wade Walker
47shailesh
Participant
Posts: 60
Joined: Tue Aug 29, 2006 11:14 pm

Post by 47shailesh »

What i got frm ur question is how create target tables based on input site names..

One Way..

$Create a table with column name "Site id", "Site Name".

$Then through a job write these table rows to a Sequential file. 'pipe' seprated

$Use a Bacth Job

$In Batch Job
Read input from the sequential file line by line.

$Use CASE and identify the "Site id" and call job instance by passing proper target table name as job parameter and so on..
$Loop until eof
wwalker
Premium Member
Premium Member
Posts: 40
Joined: Thu Mar 30, 2006 6:30 am
Location: Near Geneva, Switzerland
Contact:

Post by wwalker »

47shailesh wrote:What i got frm ur question is how create target tables based on input site names..
Thanks, but I am not sure that is quite it...

I need to create and load one table for each "site", with naming of each by parameter (for example, <schema>.<name>#SITE#)

these tables may or may not be created, depending on whether the source file contains data on a particular site.

Probably need to :
- create a list (hash, sequential) of sites,
- Read from that file, and provide distinct Site codes as a parameter to:
- i. Nested Condition Sequence (determines whether a job is run or not)
- ii. provide part of the table name

In reality, there will be +/- 25 sites....but the client wants this to be dynamic...in other words, "let's imagine there could be 20 or 1000 sites..."

So: How do I dynamically call a single job 1 to N times, based on a list of sites derived from a source file, and use the same criteria to provide a component of the table name (parameter)?
Wade Walker
47shailesh
Participant
Posts: 60
Joined: Tue Aug 29, 2006 11:14 pm

Post by 47shailesh »

What do you mean by "Distinct site codes".. and how is your source file stored..
wwalker
Premium Member
Premium Member
Posts: 40
Joined: Thu Mar 30, 2006 6:30 am
Location: Near Geneva, Switzerland
Contact:

Post by wwalker »

47shailesh wrote:What do you mean by "Distinct site codes".. and how is your source file stored..
Pretty simple, actually...a "site code" is numerical. There are different sites. So there is a number of different, distinct numerical codes that I need to extract from the file, then provide as parameters....and it is this that I am trying to resolve.

Source file will be a text file.

thanks!

W
Wade Walker
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Skip using a Sequence and just use a Batch job. You have direct control over the DS BASIC logic, you can capture a file to an array and parse it, you can directly call the job APIs, etc. For delicate logic such as yours the Sequence is the cumbersome choice.

In case you fear using a Batch job, just look at the Job Control tab under Job Properties of your Sequence. The Sequence job is just a DS BASIC program calling the public DS BASIC APIs documented in your manuals. You'll have a much easier time writing a simple DS BASIC script yourself.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
47shailesh
Participant
Posts: 60
Joined: Tue Aug 29, 2006 11:14 pm

Post by 47shailesh »

say u have 25 sites name and there corresponding Site code in written in a file smthng like:

<Site name>| <Code>
google|1
yahoo|2
AOl|3

so on...

than you can use ReadSeq to read one line at a time and store
site_name in one param and Site_code in other param..

And as you have said your job is multiple instance you can call the job using name as "jobname.":Site_code so every time a unique instance will be called...

And repat it until end of file
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

Get the list of sites into a sequential file in the first job in the sequence. Then start a loop (1 to EOF), read current loop (counter) number line from the seq file using UVactivity (Write a routine to read seq file for the counter specific line) or Execute Command activity and pass the table name to your multiple instance job.
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
Post Reply