active-to-active link problem
Moderators: chulett, rschirm, roy
active-to-active link problem
i have two transformers directly connected in the job.
if i use a sequential file or inter-process stage between them the job runs.
instead if i enable in-process or inter-process rowbuffer it does not work.
what does it mean.
can anyone throw some light.
if i use a sequential file or inter-process stage between them the job runs.
instead if i enable in-process or inter-process rowbuffer it does not work.
what does it mean.
can anyone throw some light.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
nothing to do with transformations we are just using couple of hash files along with transformer for look up.
Code: Select all
seq------transformer-------hash------transformer
| |
| |
hash |
| |
odbc----transformer-----transformer-----seq
this is the structure.
So which part as the issue, is it the two transformer coupled next to next?sendmk wrote:nothing to do with transformations we are just using couple of hash files along with transformer for look up.
Code: Select all
seq------transformer-------hash------transformer | | | | hash | | | odbc----transformer-----transformer-----seq this is the structure.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Code: Select all
seq1------transformer2-------hash2------transformer4
| |
| |
hash1 |
| |
odbc----transformer1-----transformer3-----seq2
this is the structure.
This is the actual job design. There was a problem in drawing the job structure in the previous post and did not come out correctly. Sorry.
Hash1 acts as a reference link to transformer2 and hash2 acts as a reference link to the transformer3. (Could not draw it accurately in the diagram)
When we run the job it runs forever.
When i remove only hash2 from the job it runs correctly. If i remove only hash1 from the job it runs correctly.
Is it due to deadlock between the processes of the job?
Then post a screen print of your job so it is accurate.balajisr wrote:(Could not draw it accurately in the diagram)
There are many places offering free image hosting, upload a .bmp or .jpeg there and use the Img tags to include it in your post. Then maybe we'll be able to understand the structure and flow of your job.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
this is all the information regarding the job
Hash1 acts as a reference link to transformer2 and hash2 acts as a reference link to the transformer3.
this is the structure.
Code: Select all
seq2
|
|
seq1------transformer2-------hash2------transformer4
| |
| |
hash1 |
| |
odbc----transformer1-----transformer3-----seq2
|
|
seq3
this is the structure.
Last edited by sendmk on Sun May 07, 2006 2:23 am, edited 5 times in total.
(There is no need to quote whole posts!!!)
The problem is that you have a circular dependancy in your job.
DataStage is looking at your job and a transformer2 is going to wait for the portion of the job that writes the Hash1 to complete before continuing. This is fine until at transformer3 that process is going to wait for the one starting at seq1 to complete. This is a deadlock situation.
The problem is that you have a circular dependancy in your job.
DataStage is looking at your job and a transformer2 is going to wait for the portion of the job that writes the Hash1 to complete before continuing. This is fine until at transformer3 that process is going to wait for the one starting at seq1 to complete. This is a deadlock situation.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
I thought so as well, but haven't had time to test this circular one to see if the compiler misses it. But it seems to be a logical explanation of the behaviour.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
this is all the information regarding the job
Hash1 acts as a reference link to transformer2 and hash2 acts as a reference link to the transformer3.
this is the full structure, to satisfy ur curiosity.
Code: Select all
seq2
|
|
seq1------transformer2-------hash2------transformer4------odbc2
| |
| |
hash1 |
| |
odbc1----transformer1-----transformer3-----seq2
|
|
seq3
this is the full structure, to satisfy ur curiosity.
Last edited by sendmk on Sun May 07, 2006 2:23 am, edited 1 time in total.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Actually, it's not complete, as we can't tell which way data flow with respect to the Hashed File stage hash1.
The circularity to which Arnd refers is in the Hashed File stage hash2 - it can not open its outputs until its input is closed.
Is there any reason this can't be done as two jobs - one to populate "hash2" and the other to use it?
The circularity to which Arnd refers is in the Hashed File stage hash2 - it can not open its outputs until its input is closed.
Is there any reason this can't be done as two jobs - one to populate "hash2" and the other to use it?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.