Incorrect value from DSGetLinkInfo
Moderators: chulett, rschirm, roy
Incorrect value from DSGetLinkInfo
Hi all
I have a transformer which uses the following code to increment a key for each row processed through several of the output links:
CMStart+DSGetLinkInfo (DSJ.ME,"Transform","boCM",DSJ.LINKROWCOUNT)
"Transform" is the name of the tranformer; "boCM" is the name of the first link out of the transformer, down which I send one row every time I need to increment the key.
The problem is that the key fails to increment every time a row is sent down "boCM".
The problem is, I think, related to differing constraints on output links. There are 4 links out of the transformer. Three of these share identical constraints. The fourth has a differing constraint; the data satisfying this is a subset of the data satisfying the looser constraints on the first three links.
Removing the fourth link causes the problem to disappear.
I have also tried using DSJ.ME in place of the hard coded values, and using the single input link to count on rather than the output link, neither of which has changed the result.
Any ideas?
Thanks guys
Dave
I have a transformer which uses the following code to increment a key for each row processed through several of the output links:
CMStart+DSGetLinkInfo (DSJ.ME,"Transform","boCM",DSJ.LINKROWCOUNT)
"Transform" is the name of the tranformer; "boCM" is the name of the first link out of the transformer, down which I send one row every time I need to increment the key.
The problem is that the key fails to increment every time a row is sent down "boCM".
The problem is, I think, related to differing constraints on output links. There are 4 links out of the transformer. Three of these share identical constraints. The fourth has a differing constraint; the data satisfying this is a subset of the data satisfying the looser constraints on the first three links.
Removing the fourth link causes the problem to disappear.
I have also tried using DSJ.ME in place of the hard coded values, and using the single input link to count on rather than the output link, neither of which has changed the result.
Any ideas?
Thanks guys
Dave
Every transformer output link has a localized variable called @OUTROWNUM, which is an incrementing value from 1 for every row that passes down that link. Similarly, @INROWNUM is the value for the current row count from the primary input stream.
I believe you should scrap what you're doing because it's unnecessary. @OUTROWNUM is sufficient to provide you the sequential row count for every row going down the link.
I believe you should scrap what you're doing because it's unnecessary. @OUTROWNUM is sufficient to provide you the sequential row count for every row going down the link.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
@OUTROWNUM has a different value for each link. It's misleading because you're thinking of it as @OUTROWNUM, it's internal variable name really is linkname.OUTROWNUM. So, four links, four different values, respective to the link.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Okay, wait a minute, are you trying to have a single variable common and incrementing across all of the links? If so then you should use a stage variable. Put your constraints into stage variables and do the work there. Don't be using API's ever in derivations, the overhead is a killer. You have put them into critical path. Your problem should be solvable using stage variables.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
No. If you put your constraint logic into a stage variable, it will either have a value of @TRUE or @FALSE. In another stage variable you can increment your counter if the previous stage variable evaluated to @TRUE. I'm sure you could work this out.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
You're using an API that was meant to be used for job control or process metadata extraction after a job has completed. Querying the link process metadata from within the job as it executes does not sound workable to me. I do query link process metadata from job control logic to watch a job's reference achievement during runtime, but thats externally and I know that works well.
The stage variables are the best solution, so don't lose sleep over it.![Smile :)](./images/smilies/icon_smile.gif)
The stage variables are the best solution, so don't lose sleep over it.
![Smile :)](./images/smilies/icon_smile.gif)
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There's a timing issue. The link row counts are only updated into the status table every so often.
Any reason you can't use a stage variable (as noted by others) or the @OUTROWNUM system variable (for specific (=separate) up-to-date row counts on each output link)?
Any reason you can't use a stage variable (as noted by others) or the @OUTROWNUM system variable (for specific (=separate) up-to-date row counts on each output link)?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Stage variables work great inside the transformer. However, I also use DSGetLinkInfo in the controlling job to find out how many rows are written to the database, and update my keys and config file appropriately.
The value I get returned here is sometimes correct, and sometimes not. On the occasions when it is not, it is always under, and the log also reflects that less rows have been written than is actually the case. (Querying the database reveals that DS has written 65 rows whereas the director log and DSGetLinkInfo both claim that it has only written 26.)
Is this still likely to be a timing issue, and if so is there anything I can do to get around it?
Thanks
The value I get returned here is sometimes correct, and sometimes not. On the occasions when it is not, it is always under, and the log also reflects that less rows have been written than is actually the case. (Querying the database reveals that DS has written 65 rows whereas the director log and DSGetLinkInfo both claim that it has only written 26.)
Is this still likely to be a timing issue, and if so is there anything I can do to get around it?
Thanks
This should scare you. I suspect you are updating more rows than you think. If you do not update by primary key, then you might be hitting more rows than you think.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle