Page 1 of 1

Performance while reading from hash file

Posted: Sat Feb 12, 2005 4:06 pm
by SonShe
I have a job that has been designed as below:

Oracle Stage ---> Transformer ----> Transformer ----> Transformer ----> HashFile ------> Transformer ---> Transformer --- > Oracle Stage.

These various transformers have look-ups from hash files. When I run the job, data moves from the first Oracle stage through the transformers to the hash file. Until all the data are written to the hash file, the read process from the hash file to the final Oracle stage does not start.

This job has Inter-process buffer enabled at the job level. The unix server has 2 CPUs. My understanding is that at least 2 processes can run simultaneously with 2 CPUs. I am of the impression that the first OCI stage to the hash file is one process and from the hash file to the destination OCI stage is the second process. SO why are not these processes running at the same time?

I will appreciate any help in this regard.

Thanks.

Posted: Sat Feb 12, 2005 4:12 pm
by kcbland
Because that's not how it works. What it does is allow multiple rows to be in transit across a string of active stages between two passive stages, as opposed to a single row having to make it all of the way across to the second passive stage before the next row can start.

Your job is working perfectly. Obviously, someone put the hash file there for a reason, most likely because they either wanted to reference it while loading it, or because they're using it as a cheap duplication reduction device. You should double check the jobs requirements before removing that hash file.

What is happening is that each transformer between the first OCI and hash file is operating as its own process and each process is allowed to use different cpus.