Multiple instances in parllel jobs

[UPS]

Hi
I have a job that extracts data from a table in oracle database and puts it into a dataset.this job takes the a parmeter 'period' and runs the job.how can I run 'n' number of these jobs (all these jobs should be running at time not one at a time) each taking a different parametre value.


in other words how can I run multiple instances in parllel jobs

[kris]

in other words how can I run multiple instances in parllel jobs

In the designer, in the Job Properties tab, select allow multiple instance.

That will allow you to run an instance of the job using invocation id. But you have to be careful as you are talking about instanciating a parallel job. Make sure you do not overload your server.

Best regards,

[kris]

in other words how can I run multiple instances in parllel jobs

In the designer, in the Job Properties tab, select allow multiple instance.

That will allow you to run an instance of the job using invocation id. But you have to be careful as you are talking about instanciating a parallel job. Make sure you do not overload your server.

Best regards,

[DSguru2B]

Make the job a multiple instance job and make sure that you incorporate the instance name/number as part of the dataset name to ensure that each instance has a unique dataset to write to.
Fire off each instance at the same time using a sequence job or an enterprise scheduler. Does that answer your query or you are looking for something else?

[anandkumarm]

Hi
THank you for the reply...i dont think using a sequencer will help me in my situation coz i want all the instances to be running at atime and in a sequencer if you use a loop they will be executed one after the other...please do correct me if i am wrong....
and about the invocation id can you be more specific about passing the invoaction id...in parllel jobs....

[ArndW]

You are correct in that the Job Activity will issue a wait and not continue until the called job has finished. If you don't care how many instances are run concurrently, replace the Job Activity stage with a Command stage and build the command line for dsjob manually in the sequence, but don't add the '-wait' option.

[ray.wurlod]

You don't need a loop - just multiple Job activities each referring to the same multi-instance job but with different invocation IDs.

<irony>Create as many as you like - everyone knows that a DataStage server has an infinite supply of CPU, memory, disk and network resources!</irony>

[bcarlson]

One issue we have run into when running multiple instances is that if you are trapping the logs on the server side (ex. you have a Unix script that runs the DS job and captures logging), the logging traps logs for all instances that are running at that time.

Say I have 3 instances running at the same time. I may try to trap the logs for each in their own individual logs. However, what I end up with is each log containing the logs of the other 2 instances. I am not sure why this is happening.

In Director, you will see that instances are color coordinated and you can easily identify one instance from another. For some reason, this does not work on the log retrieval side (sorry, not on my work laptop, so I don't have access to the documentation adn don't remember the name of the Unix program for retrieving logs - is that dsjob or something else?).

Now, I will put in a disclaimer. It is entirely possible that it is our process for pulling logs that is not working correctly. If someone else has run into this problem, let me know. If they have not and are able to easily retrieve logs to a Unix file for a specific instance, LET ME KNOW!

Brad.

[ray.wurlod]

I don't know of any way to do it with the API (that is, dsjob or one you wrote yourself). It would, of course, be possible using DataStage/SQL; for example

Code: Select all

$DSHOME/bin/dssh "SELECT columns FROM RT_LOGnnn WHERE ..." > file

I too don't have DataStage readily available at the moment, hence the sparse information in the above.

[gaurav_shukla]

We can make job multiple instance by checking the Multiple Instance Check button in Job Properties window.

Now you need to run the job through unix script with parameter as your period for which you want to run the job.

And this period will also be your invocation id.

Let me know this works or not.

[anandkumarm]

Hi
THank you for the reply...I want to know how the job takes the period as the invocation id when we are not specifing anything about it saying the period is the invocation id also...
dsjob -run -param period='june-05' projectname jobname

is there anyother command for passing the invocation id also

[mujeebur]

Try this :

dsjob -run -param period='june-05' projectname jobname.period