Incorrect value from DSGetLinkInfo

freef1yer · Post by **freef1yer** » Wed Nov 05, 2003 1:31 pm

Hi all

I have a transformer which uses the following code to increment a key for each row processed through several of the output links:

CMStart+DSGetLinkInfo (DSJ.ME,"Transform","boCM",DSJ.LINKROWCOUNT)

"Transform" is the name of the tranformer; "boCM" is the name of the first link out of the transformer, down which I send one row every time I need to increment the key.

The problem is that the key fails to increment every time a row is sent down "boCM".

The problem is, I think, related to differing constraints on output links. There are 4 links out of the transformer. Three of these share identical constraints. The fourth has a differing constraint; the data satisfying this is a subset of the data satisfying the looser constraints on the first three links.

Removing the fourth link causes the problem to disappear.

I have also tried using DSJ.ME in place of the hard coded values, and using the single input link to count on rather than the output link, neither of which has changed the result.

Any ideas?

Thanks guys

Dave

kcbland · Post by **kcbland** » Wed Nov 05, 2003 1:35 pm

Every transformer output link has a localized variable called @OUTROWNUM, which is an incrementing value from 1 for every row that passes down that link. Similarly, @INROWNUM is the value for the current row count from the primary input stream.

I believe you should scrap what you're doing because it's unnecessary. @OUTROWNUM is sufficient to provide you the sequential row count for every row going down the link.

freef1yer · Post by **freef1yer** » Wed Nov 05, 2003 1:42 pm

The problem with @OUTROWNUM is that it returns the number of rows going out of that specific link. I have 4 links, one of which has less rows going out of it than the other 3. So @OUTROWNUM will not return me the correct value for this link.

kcbland · Post by **kcbland** » Wed Nov 05, 2003 1:50 pm

@OUTROWNUM has a different value for each link. It's misleading because you're thinking of it as @OUTROWNUM, it's internal variable name really is linkname.OUTROWNUM. So, four links, four different values, respective to the link.

kcbland · Post by **kcbland** » Wed Nov 05, 2003 1:53 pm

Okay, wait a minute, are you trying to have a single variable common and incrementing across all of the links? If so then you should use a stage variable. Put your constraints into stage variables and do the work there. Don't be using API's ever in derivations, the overhead is a killer. You have put them into critical path. Your problem should be solvable using stage variables.

freef1yer · Post by **freef1yer** » Wed Nov 05, 2003 2:35 pm

Yes, the idea is to have a single common variable and increment it across all of the links.

Thanks for the tip re: performance, I'll put everything into stage variables. I'll still need to use the DSGetLinkInfo function though within the stage variable?

kcbland · Post by **kcbland** » Wed Nov 05, 2003 3:09 pm

No. If you put your constraint logic into a stage variable, it will either have a value of @TRUE or @FALSE. In another stage variable you can increment your counter if the previous stage variable evaluated to @TRUE. I'm sure you could work this out.

freef1yer · Post by **freef1yer** » Thu Nov 06, 2003 5:52 am

Yep, that worked well. I still don't really understand why the DSGetLinkInfo function wasn't returning me the values I was expecting, but the job now works fine.

Thanks

Dave

kcbland · Post by **kcbland** » Thu Nov 06, 2003 7:03 am

You're using an API that was meant to be used for job control or process metadata extraction after a job has completed. Querying the link process metadata from within the job as it executes does not sound workable to me. I do query link process metadata from job control logic to watch a job's reference achievement during runtime, but thats externally and I know that works well.

The stage variables are the best solution, so don't lose sleep over it.

ray.wurlod · Post by **ray.wurlod** » Thu Nov 06, 2003 4:02 pm

There's a timing issue. The link row counts are only updated into the status table every so often.
Any reason you can't use a stage variable (as noted by others) or the @OUTROWNUM system variable (for specific (=separate) up-to-date row counts on each output link)?

freef1yer · Post by **freef1yer** » Fri Nov 07, 2003 11:53 am

Stage variables work great inside the transformer. However, I also use DSGetLinkInfo in the controlling job to find out how many rows are written to the database, and update my keys and config file appropriately.

The value I get returned here is sometimes correct, and sometimes not. On the occasions when it is not, it is always under, and the log also reflects that less rows have been written than is actually the case. (Querying the database reveals that DS has written 65 rows whereas the director log and DSGetLinkInfo both claim that it has only written 26.)

Is this still likely to be a timing issue, and if so is there anything I can do to get around it?

Thanks

kcbland · Post by **kcbland** » Fri Nov 07, 2003 12:03 pm

This should scare you. I suspect you are updating more rows than you think. If you do not update by primary key, then you might be hitting more rows than you think.

freef1yer · Post by **freef1yer** » Fri Nov 07, 2003 12:11 pm

The job inserts rows, it doesn't update any. It inserts one row into a number of tables for each one input row.

The destination database tables each contain the correct amount of rows and the correct data. However, the log reports that _less_ rows have been written than is actually the case.

kduke · Post by **kduke** » Fri Nov 07, 2003 2:02 pm

I think your problem is you are getting info on the current running job. I have had a lot of problems doing that. If you check from an extern process then it works. Those routines were not designed to work on the active job.

Kim.