Multiple writes to a Hash file

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
Neil C
Participant
Posts: 46
Joined: Sun Apr 06, 2003 8:59 pm
Location: Auckland, New Zealand

Multiple writes to a Hash file

Post by Neil C »

I have a DS job that has three transform stages, all writing to the same hash file. The job exceute SQL to access the MAX(BATCH_ID) from a number of Oracle tables. The first stage accesses a single table and completes in a second or so, and writes this row to the hash file after a delete/create of the hash file.

The create is defaulting to WRITE DEFERRED caching (because I did not think to question the wisdom or otherwise of this). There is a message in the log saying that WRITE DEFERRED caching is dissabled. Why?

The the other two stages eventually write about 100 rows each to the hash file, after maybe 15-20 minutes elapsed.

My question is, what will happen if these two stages were to start writing to the hash file simultaneously. Is there any locking mechanism used with hash files that will prevent or allow concurrent writes to the same hash file? Should the caching be defined as NONE, or WRITE IMMEDIATE, in the definition options?

(I have seen two individual jobs running where the first was reading a hash file, then the second started later, deleted/created the hash file and wrote to it, hence destroying the data that the first job thought was there! These jobs now use different names. It was not easy to find that bug!).

TIA,
Neil Courtney
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

What's the setting of the uvconfig parameters DISKCACHE and DCWRITEDAEMON? Use CONFIG DATA to find out. Read the comments in uvconfig to ascertain what the effect of each is.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Neil C
Participant
Posts: 46
Joined: Sun Apr 06, 2003 8:59 pm
Location: Auckland, New Zealand

Post by Neil C »

DISKCACHE = -1 and DCWRITEDEAMON = 0.

Off to do some reading....

Thanks Ray.
Post Reply