Routines - When must they be recompiled?

[tgmedear]

Greetings, I have experienced multiple events where a DataStage routine produced incorrect results in a DataStage job. I then recompiled the routine, and everything ran fine. I thought I had found the solution when I started recompiling all routines and jobs before execution using the muptiple job compile functionality, however, the problem has reappeared. Has anyone experienced this? How do I get around this problem? Under what conditions must DS routines and jobs be recompiled? Has anyone experienced situations where the DS Server inexplicably executed an old version of a DS Routine? Thank You.

[ray.wurlod]

Welcome aboard.

Please give examples. Recompiling a Routine overwrites the old object code, so the only way it could be executed is by a process that was already using it when the Routine was recompiled.

You only need to recompile a Routine if its source code is changed.

You should only recompile a Routine if it is not in use as the using process will have loaded the (old) object code into memory, and may write it back to disk when done.

[tgmedear]

Thank you for your interest in my problem. I have the following routine:

FUNCTION getIataIdk(iataKey, tvlagtpntreq)
IF len(iataKey) > 0 THEN
Ans = iataKey
END
ELSE if len(tvlagtpntreq) > 0 THEN
Ans = tvlagtpntreq
end
else
Ans = ""
END;

After running a DS sequence that calls a DS job that calls this routine, I noticed the the value being returned was incorrectly "". I opened the routine and tested it, and sure enough, it gave the wrong results "". I then recompiled the routine and tested again, and it worked correctly. I did not change the code.

The DS job is used in several sequences. Could it be that the sequence job executable had the wrong executable stored in it and overwrote the correct executable?

Thank you.

[ray.wurlod]

No. Routine code is stored separately from job/sequence code.

Can you reliably produce incorrect results in test mode? What other things are happening on the server when you can? In particular, might any other process be using this routine?

Please advise what test data gave incorrect results. Double click on the Results cell on the test grid and post the contents thereof.

Any incoming NULL on the arguments will return "" based on your logic.

[tgmedear]

No. Unfortunately, I can not reliably produce the problem. I recompiled the DS job and all the sequences involved, and it appears everything is running correctly, now. But I did that before and the problem came back. I understand that it sounds crazy, but it has happened multiple times, now.

[ray.wurlod]

Well, you know what to trap and post next time it occurs.

[tgmedear]

I have figured out the cause of this problem. I have one routine in folder A named getIataIdk and another routine in folder B named getIATAidk. When these two routines are compiled, it appears that the executable of one covers over the executable of the other. Basically, DataStage allows different routines with the same name but different case letters to be created, but it appears to not differentiate between the two when compiled. Perhaps this is resolved in a newer version of DS. I am running on 7.5.1.

[chulett]

Ah, yes... Windows. DataStage forces object names to be case sensitive and Windows could case less, happily letting one executable overwrite the other.

Nothing really to fix here other than your naming conventions, it seems.

[tgmedear]

Well, Windows is smart enough to know that c:\A\getIata.txt is not c:\B\getIata.txt. DS should be able to do the same.

[chulett]

There's no smarts there - those are two completely different paths. On any operating system. What you are seeing is one of the consequences of a 'caseless' filesystem. DataStage has no control over that aspect of its existence on Windows and can't store two files with 'the same name' (where they only differ by case) in the same place.

Well, actually it can as you've found out. Problem is it is much like a hashed file - destructive overwrite. Last one in wins.

Change the name of one of your routines - even just adding a '2' at the end or something of that nature will let them live together, happily ever after.

[tgmedear]

I disagree. If DataStage is smart enough to know the difference between the text of the two different routines, it should be able to handle the different executables. Instead, the executable of one is covering the other. In particular, since the files are in different categories, DataStage should handle this. Windows allows files in different paths with the same file name. I do not agree this is a Windows issue. Regardless, I have changed the name of one of the routines, so I've got things working. But this is bad. There is no warning of the error. What if I accidentally duplicate the name of a routine provided by DataStage? There appears no way to know it.

[ray.wurlod]

Routines are stored in a hashed file called DS_ROUTINES. The routine name is the primary key. Hashed files are updated via destructive overwrite.

It should, as a result, be impossible to have two routines of the same name in the same project. Category is a non-key field in the Routine's record.

I think you need to check the internal integrity of your DS_ROUTINES hashed file, using fixtool or uvfixfile in report mode.

[tgmedear]

I may have misunderstood you. My point is that since I have two different routines in two different categories, and DataStage is handling their text existence, it should handle their separate executables.

[ray.wurlod]

My point is that is does (keep their executables separate).

The routines must have different names. Their different executables are eponymous files in the DSU_BP.O subdirectory in your project.

You simply can not have two routines of the same name in the same project. Prove this with the following query:

Code: Select all

SELECT DSRID, COUNT(*) FROM DS_ROUTINES GROUP BY DSRID HAVING COUNT(*) > 1;

Or prove to me that you do.

Code: Select all

SELECT DSRID, CATEGORY FROM DS_ROUTINES WHERE DSRID = '<<RoutineName>>';

If the routines have the same name but different casing, the fault lies with the Windows operating system for not preserving case, not with DataStage, which does preserve case. The following query will perform a case-insensitive search.

Code: Select all

SELECT DSRID, CATEGORY FROM DS_ROTUINES WHERE EVAL "UPCASE(DSRID)" = EVAL "UPCASE('<<RoutineName>>')" ;

If you find multiples here, all complaints should go to billg@microsoft.com.

[ArndW]

Ray, the source code is case sensitive in the DS_ROUTINES hashed file, but the executables are in the BP.O type 19 directory (I can't recall the actual dir name right now) and thus subject to the DOS case limitations.