Running job in MI
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 73
- Joined: Wed Jun 28, 2006 3:27 pm
- Location: NJ
Running job in MI
I wanted to know how to use Multiple Instance in a job or when to check the box ?
How the job will run and what is the importance of invocation ids ?
I have bit idea but as I am a rookie I need more help to go. Please through some light on this.
I am asking this becoz I have developed the job which runs too slow, so just wondering can I run it in Multiple instance... Job is like :
OCI------Transformer----[ 15 Hashed files ]-------Transformer-----Seqfile
Thanx
How the job will run and what is the importance of invocation ids ?
I have bit idea but as I am a rookie I need more help to go. Please through some light on this.
I am asking this becoz I have developed the job which runs too slow, so just wondering can I run it in Multiple instance... Job is like :
OCI------Transformer----[ 15 Hashed files ]-------Transformer-----Seqfile
Thanx
"Nobody is expert in Everything,
But Everybody is expert in Something."
But Everybody is expert in Something."
You can only write to a sequential file from one process at a time. You might change your job around so the multiple instances write to the hashed files simultaneously, and then one more job to run single-instance to write from the hashed files to the sequential one.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 73
- Joined: Wed Jun 28, 2006 3:27 pm
- Location: NJ
Sorry I didnt mention, I am only reading from hashed files and writing to seq file.
Moreover there is only one Transformer in the job.( Thats why I used the same name )
As as you said only one process writes in seq file, I can write after job subroutine with different file names appended by some ID.
Please suggest..
Moreover there is only one Transformer in the job.( Thats why I used the same name )
As as you said only one process writes in seq file, I can write after job subroutine with different file names appended by some ID.
Please suggest..
"Nobody is expert in Everything,
But Everybody is expert in Something."
But Everybody is expert in Something."
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You missed Arnd's point. The operating system will only allow one process at a time to write to a file. Therefore if you want multi-instance, the job invocations must write to separate files.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 73
- Joined: Wed Jun 28, 2006 3:27 pm
- Location: NJ
Ok so now how shall I do that ? Because I dont know how many instances it will need. I will have to do trial and error to minimize the run time till its affordable enough.
My point was can I use some thing like using ExeSh as after job subroutine which will create each different file for each instance.( shell script which appends filename to invocation ID )
My point was can I use some thing like using ExeSh as after job subroutine which will create each different file for each instance.( shell script which appends filename to invocation ID )
"Nobody is expert in Everything,
But Everybody is expert in Something."
But Everybody is expert in Something."
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 73
- Joined: Wed Jun 28, 2006 3:27 pm
- Location: NJ
Hmm. Thanks ray.
But as I have never done this before what should my primary steps ? Suppose I want to run the job in 5 instances then how shall I proceed , do I need to write some script or manually I have to run the job 5 times passing 5 different invocation ids each?
But as I have never done this before what should my primary steps ? Suppose I want to run the job in 5 instances then how shall I proceed , do I need to write some script or manually I have to run the job 5 times passing 5 different invocation ids each?
"Nobody is expert in Everything,
But Everybody is expert in Something."
But Everybody is expert in Something."
The latter, i.e, run the job five times with five different Invocation Ids.
Consequentily, you can write your result to a hashed file whos key can be @INROWNUM coupled with Invocation ID. This way all five instances will write to the same hashed file and each record will be preserved due to a different key. A second job can copy the hashed file to a sequential file.
Consequentily, you can write your result to a hashed file whos key can be @INROWNUM coupled with Invocation ID. This way all five instances will write to the same hashed file and each record will be preserved due to a different key. A second job can copy the hashed file to a sequential file.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
No, you can change the job that runs multi-instance to just write to the hashed files. Run that with as many parallel instances as your heart desires. Once all instances are finished, start one job to read the hashed files and write to the sequential file.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 73
- Joined: Wed Jun 28, 2006 3:27 pm
- Location: NJ
Thankyou so much guys. So let me summarize the points and then I will mark this issue resolved-
Design 2 jobs
1st job
-define parameters(invocation ids)
-enable MI
-write all intances to single hashed file with key @INROWNUM=invocation ids
-Run this same job 5 times continuously without waiting for previous intance to complete(if there are 5 instances)
2nd job
-copy hashed file to seq file
Design 2 jobs
1st job
-define parameters(invocation ids)
-enable MI
-write all intances to single hashed file with key @INROWNUM=invocation ids
-Run this same job 5 times continuously without waiting for previous intance to complete(if there are 5 instances)
2nd job
-copy hashed file to seq file
"Nobody is expert in Everything,
But Everybody is expert in Something."
But Everybody is expert in Something."