Internal File Corruption Detected

Zinna · Post by **Zinna** » Wed Oct 30, 2002 7:04 pm

Please Help!

I cannot seem to completed delete and recreate a particular hash file. I have used DELETE.FILE filename, all seems successful, D_filename directory completed gone, all traces in the NT file system seem to have been deleted. VOC entry is all. All seems sweet.

I validate the file, all looks fine. But when I try to open the file or perform a select * from filename in administrator, i get a "Read Operation Failure. Message[00000]Internal File Corruption Detected".

How can I repair this? By the way I have tried uvfixfile -f filename without success. Assistance greatly appreciated.

Zinna

ray.wurlod · Post by **ray.wurlod** » Wed Oct 30, 2002 7:12 pm

A hashed file consists of three parts; its VOC entry, its data portion and its dictionary portion. The data portion is a file system object with the same name as the hashed file (usually), and the dictionary portion is a file system object having that name preceded by "D_".
Once you have deleted the VOC entry, the only way to delete the hashed file, so it can be re-created, is either to re-create the VOC entry or to remove the data and dictionary portions from the file system.
The data portion is usually a directory containing three files, so you should specify the /S and /Q switches if using the DEL command to remove them. The dictionary portion is usually a file.
It is perfectly acceptable, if the VOC entry does not exist, to remove them using Explorer.

ray.wurlod · Post by **ray.wurlod** » Wed Oct 30, 2002 7:16 pm

Another possibility, triggered by seeing the message id 00000, is that the system message file (SYS.MESSAGE) is the one in which internal corruption has been detected. I would recommend using uvfixfile (without the -fix option) to verify whether this is the case.
If so, you might attempt to repair SYS.MESSAGE, but be prepared to delete it and restore from backup if this turns out not to be successful. Since every DataStage process has SYS.MESSAGE open, you should execute the repair from the operating system (a DOS shell, in your case) with no DataStage processes operating.

Zinna · Post by **Zinna** » Wed Oct 30, 2002 9:57 pm

Yes, when I use the DELETE.FILE command, it reports it has successfully deleted all three parts. And indeed when I checked the VOC and NT Explorer it no longer exists. Indicating that it has deleted all traces.

When I then recreate it (using validation) it works, but when I then try to view/select from the table that is when it says there is internal corruption. It works fine when I use another filename, so it seems there is still a record of the old filename somewhere in the internal architecture/UV tables?

A hashed file consists of three parts; its VOC entry, its data portion and its dictionary portion. The data portion is a file system object with the same name as the hashed file (usually), and the dictionary portion is a file system object having that name preceded by "D_".
Once you have deleted the VOC entry, the only way to delete the hashed file, so it can be re-created, is either to re-create the VOC entry or to remove the data and dictionary portions from the file system.
The data portion is usually a directory containing three files, so you should specify the /S and /Q switches if using the DEL command to remove them. The dictionary portion is usually a file.
It is perfectly acceptable, if the VOC entry does not exist, to remove them using Explorer.

ray.wurlod · Post by **ray.wurlod** » Wed Oct 30, 2002 10:43 pm

How do you create it, and what - exactly - is the error message?
For example, if you'd created it with CREATE TABLE there would be entries in the system tables (UV_TABLES, UV_COLUMNS, UV_USERS and possibly UV_ASSOC) recording its existence. Do you check that the entry truly has been deleted from the VOC file?

Zinna · Post by **Zinna** » Wed Oct 30, 2002 11:19 pm

I create it by Validation in DS Director. The EXACT error message is "Read Operation Failure. Message[00000]Internal File Corruption detected".

And YES, after deleting I do check the VOC... using SELECT * FROM VOC WHERE NAME='filename';

I suspect there is another system file where the deleted filename is retained?

How do you create it, and what - exactly - is the error message?
For example, if you'd created it with CREATE TABLE there would be entries in the system tables (UV_TABLES, UV_COLUMNS, UV_USERS and possibly UV_ASSOC) recording its existence. Do you check that the entry truly has been deleted from the VOC file?

ray.wurlod · Post by **ray.wurlod** » Thu Oct 31, 2002 1:06 am

Can you please show us the text of the logged message when the hashed file is created, so we can see whether CREATE.FILE or mkdbfile is the command used?
Unfortunately for your theory, there is no other place that information about files is stored (unless you've created secondary indices or made the hashed file a part of a Distributed file). So it seems possible that the file with the internal corruption may be VOC (yuk!) - perhaps you could check VOC with uvfixfile (without the -fix option).

Zinna · Post by **Zinna** » Thu Oct 31, 2002 4:12 pm

Hi Ray,

I ran the uvfix VOC and got the following result...
------------------------
Beginning TRACE of VOC.
TRACE of VOC completed.

Scanning overflow buffers.
Scan complete.

23 group(s) processed.
192 group buffer(s) processed.
4711 record(s) processed.
Number of data bytes = 364268.
-----------------------------------

What do you think? Doesn't seem suspicious.

When I validate the job, to create the hash file it use CREATE.FILE. Here is the log entry from DS Director:

--------------------------------------
testzc.h_deal_ld.output: Creating File h_deal_ld, Executing Command =
CREATE.FILE h_deal_ld DYNAMIC
---------------------------------

Once created, get the error (noted in previous reply) when I try to perform a select on the file.

When I run the job, I get the following error message in the director job log:

----------------------------------------
DataStage Job 453 Phantom 694
Program "JOB.477117071.DT.1272336651.V0S2": Line 293, Read operation failure. Message[000000]Program "JOB.477117071.DT.1272336651.V0S2"

Computed blink of 0x800 does not match expected blink of 0x0!
Detected within group starting at address 0x3000!
Program "JOB.477117071.DT.1272336651.V0S2": Line 293, Internal data error.
File 'D:ARDENTDATASTAGEPROJECTSEDW_DEVL/h_deal_ld/DATA.30'
-----------------------------------------

Do you know what "blink" is?

Thanks again,
Zinna

ray.wurlod · Post by **ray.wurlod** » Thu Oct 31, 2002 4:41 pm

BLINK stands for "backward link". A BLINK error is a fairly serious kind of error, and one which definitely should not occur in a newly-created (and therefore empty) hashed file. It relates to the way in which groups are strung together (and you only have one group!).
The detailed error message (thanks for providing that) indicates that it is in your hashed file, not in VOC (whew!).
Try clearing the file (CLEAR.FILE filename) to see whether that causes the problem to vanish.
If not, try repairing the file with uvfixfile (with the -fix option, for which you will need administrator privilege).
Finally, try creating a hashed file with a different name by exactly the same means, without deleting the corrupt one first. You may be hitting a bad spot on disk, or something like that.
If all of those fail, report the fact that you are getting a BLINK error in a newly-created hashed file to your vendor or Ascential; there's something really wrong here. I have tried to reproduce your problem on DS 5.2 but could not.

ray.wurlod · Post by **ray.wurlod** » Thu Oct 31, 2002 4:52 pm

Another thing to check.
Earlier you reported that you got the message attempting to query the hashed file. Was this immediately after validating the job, or after running the job?
What happens (exact error message please) if you query the hashed file immediately after validating the job?

Zinna · Post by **Zinna** » Thu Oct 31, 2002 5:35 pm

CLEAR.FILE filename worked!!!!

Thanks Ray!

Cheers,
Zinna