Multiple instances in parllel jobs
Moderators: chulett, rschirm, roy
Multiple instances in parllel jobs
Hi
I have a job that extracts data from a table in oracle database and puts it into a dataset.this job takes the a parmeter 'period' and runs the job.how can I run 'n' number of these jobs (all these jobs should be running at time not one at a time) each taking a different parametre value.
in other words how can I run multiple instances in parllel jobs
I have a job that extracts data from a table in oracle database and puts it into a dataset.this job takes the a parmeter 'period' and runs the job.how can I run 'n' number of these jobs (all these jobs should be running at time not one at a time) each taking a different parametre value.
in other words how can I run multiple instances in parllel jobs
Re: Multiple instances in parllel jobs
UPS wrote: in other words how can I run multiple instances in parllel jobs
In the designer, in the Job Properties tab, select allow multiple instance.
That will allow you to run an instance of the job using invocation id. But you have to be careful as you are talking about instanciating a parallel job. Make sure you do not overload your server.
Best regards,
~Kris
Re: Multiple instances in parllel jobs
UPS wrote: in other words how can I run multiple instances in parllel jobs
In the designer, in the Job Properties tab, select allow multiple instance.
That will allow you to run an instance of the job using invocation id. But you have to be careful as you are talking about instanciating a parallel job. Make sure you do not overload your server.
Best regards,
~Kris
Make the job a multiple instance job and make sure that you incorporate the instance name/number as part of the dataset name to ensure that each instance has a unique dataset to write to.
Fire off each instance at the same time using a sequence job or an enterprise scheduler. Does that answer your query or you are looking for something else?
Fire off each instance at the same time using a sequence job or an enterprise scheduler. Does that answer your query or you are looking for something else?
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Premium Member
- Posts: 55
- Joined: Tue Feb 24, 2004 8:17 am
Hi
THank you for the reply...i dont think using a sequencer will help me in my situation coz i want all the instances to be running at atime and in a sequencer if you use a loop they will be executed one after the other...please do correct me if i am wrong....
and about the invocation id can you be more specific about passing the invoaction id...in parllel jobs....
THank you for the reply...i dont think using a sequencer will help me in my situation coz i want all the instances to be running at atime and in a sequencer if you use a loop they will be executed one after the other...please do correct me if i am wrong....
and about the invocation id can you be more specific about passing the invoaction id...in parllel jobs....
You are correct in that the Job Activity will issue a wait and not continue until the called job has finished. If you don't care how many instances are run concurrently, replace the Job Activity stage with a Command stage and build the command line for dsjob manually in the sequence, but don't add the '-wait' option.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You don't need a loop - just multiple Job activities each referring to the same multi-instance job but with different invocation IDs.
<irony>Create as many as you like - everyone knows that a DataStage server has an infinite supply of CPU, memory, disk and network resources!</irony>
<irony>Create as many as you like - everyone knows that a DataStage server has an infinite supply of CPU, memory, disk and network resources!</irony>
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
One issue we have run into when running multiple instances is that if you are trapping the logs on the server side (ex. you have a Unix script that runs the DS job and captures logging), the logging traps logs for all instances that are running at that time.
Say I have 3 instances running at the same time. I may try to trap the logs for each in their own individual logs. However, what I end up with is each log containing the logs of the other 2 instances. I am not sure why this is happening.
In Director, you will see that instances are color coordinated and you can easily identify one instance from another. For some reason, this does not work on the log retrieval side (sorry, not on my work laptop, so I don't have access to the documentation adn don't remember the name of the Unix program for retrieving logs - is that dsjob or something else?).
Now, I will put in a disclaimer. It is entirely possible that it is our process for pulling logs that is not working correctly. If someone else has run into this problem, let me know. If they have not and are able to easily retrieve logs to a Unix file for a specific instance, LET ME KNOW!
Brad.
Say I have 3 instances running at the same time. I may try to trap the logs for each in their own individual logs. However, what I end up with is each log containing the logs of the other 2 instances. I am not sure why this is happening.
In Director, you will see that instances are color coordinated and you can easily identify one instance from another. For some reason, this does not work on the log retrieval side (sorry, not on my work laptop, so I don't have access to the documentation adn don't remember the name of the Unix program for retrieving logs - is that dsjob or something else?).
Now, I will put in a disclaimer. It is entirely possible that it is our process for pulling logs that is not working correctly. If someone else has run into this problem, let me know. If they have not and are able to easily retrieve logs to a Unix file for a specific instance, LET ME KNOW!
Brad.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I don't know of any way to do it with the API (that is, dsjob or one you wrote yourself). It would, of course, be possible using DataStage/SQL; for example
I too don't have DataStage readily available at the moment, hence the sparse information in the above.
Code: Select all
$DSHOME/bin/dssh "SELECT columns FROM RT_LOGnnn WHERE ..." > file
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 12
- Joined: Wed Jun 13, 2007 2:12 am
Re: Multiple instances in parllel jobs
We can make job multiple instance by checking the Multiple Instance Check button in Job Properties window.
Now you need to run the job through unix script with parameter as your period for which you want to run the job.
And this period will also be your invocation id.
Let me know this works or not.
Now you need to run the job through unix script with parameter as your period for which you want to run the job.
And this period will also be your invocation id.
Let me know this works or not.
-
- Premium Member
- Posts: 55
- Joined: Tue Feb 24, 2004 8:17 am
Re: Multiple instances in parllel jobs
Hi
THank you for the reply...I want to know how the job takes the period as the invocation id when we are not specifing anything about it saying the period is the invocation id also...
dsjob -run -param period='june-05' projectname jobname
is there anyother command for passing the invocation id also
THank you for the reply...I want to know how the job takes the period as the invocation id when we are not specifing anything about it saying the period is the invocation id also...
dsjob -run -param period='june-05' projectname jobname
is there anyother command for passing the invocation id also
Re: Multiple instances in parllel jobs
Try this :
dsjob -run -param period='june-05' projectname jobname.period
dsjob -run -param period='june-05' projectname jobname.period