Urgent: Project has been corrupted

vinodlakshmanan · Post by **vinodlakshmanan** » Thu May 12, 2005 6:35 am

I have an existing project. There was some problem on the server due to which the project was corrupted. I get the error "CANNOT OPEN FILE DS_JOBOBJECTS". I can create and save a new job, but cannot open it for re-editing and cannot delete any jobs. I have tried re-indexing the entire project using DS.REINDEX ALL, but it does not seem to work.
I have tried clearing the jobobjects file so that it may re-create the file during re-indexing (don't know if it was correct, was a last desperate measure), but I got the following error:

>CLEAR.FILE DS_JOBOBJECTS
WARNING: Internal file corruption detected during file open!
File must be repaired, possible truncation.
hsize: 2048
bsize: 2048
fsize: 1915904
Cannot open file "DS_JOBOBJECTS".
*** Processing cannot continue. ***

UVFIXFILE also does not work, see below:

>UVFIXFILE DS_JOBOBJECTS FIX

User does not have superuser priviledges!
Disabling the FIX option and continuing.
Warning: "DS_JOBOBJECTS/OVER.30" has been truncated.
WARNING: Internal file corruption detected during file open!
File must be repaired, possible truncation.
hsize: 2048
bsize: 2048
fsize: 1915904
Warning: Unable to set GROUP LOCKS on DS_JOBOBJECTS.

Beginning TRACE of DS_JOBOBJECTS.

ERROR: Processing Primary group 25, Overflow group 0.
Bad forward link found at address 0x0000ccf0.
TRACE of DS_JOBOBJECTS completed.

Warning: file load discrepancy detected.

Scanning free buffer chain.
Scan complete.

Scanning overflow buffers.
Warning: No link to overflow group 122.
Warning: No link to overflow group 123.
Warning: No link to overflow group 924.
Warning: No link to overflow group 934.
Scan complete.

280 group(s) processed.
431 group buffer(s) processed.
1340 record(s) processed.
Number of data bytes = 459932.

Cleanup of project also does not work, it gives the same error as during clearing the file.

Now, can I fix the file using UVFIXFILE, I do not know what is the admin user-name (and maybe default passwd) is so please help me out.
If this is not possible, can I delete the file and copy a DS_JOBOBJECTS of some other project, then will re-indexing re-create the job-objects file?

Await someon'es quick response.

Vinod

ArndW · Post by **ArndW** » Thu May 12, 2005 6:58 am

Vinod,

clearing the DS_JOBOBJECTS will not help you. Can you login with sufficient access to actually do the UVFIXFILE (that is why the FIX option didn't do anything)?

This file cannot be rebuilt from another project, unless it is a mirror copy.

Before continuing on this thread, do the UVFIXFILE with the FIX option and tell us what happened. The OVER.30 file will contain significantly less data than the data file, so any losses of data might not be too critical for you.

This might just be the first problem of many - do you know what happened to cause this file to get corrupted?

vinodlakshmanan · Post by **vinodlakshmanan** » Thu May 12, 2005 7:07 am

Hi ArnD,

I tries logging on using DataStage Administrator and also using uv from the Projects/<proj name> directory, however FIX does not work. Is there any default user-name for admin (like scott/tiger) or is it set during installation for which I will have to contact the admin?
I do not know why this problem occured, the problem just occured one fine day. It corrupted arbitrary files of all open projects, sme projects had their log files corrupted, this project had the JOBOBJECTS file corrupted and so on. The UNIX server was restarted, but the damage was already done.

ArndW · Post by **ArndW** » Thu May 12, 2005 7:12 am

Vinod,

go into the project directory in UNIX and see who the file owner is and access rights are - perhaps it is root. It might be best to make a copy of the directory & it's contents from UNIX before doing the FIX option, just in case it truncates more than it ought to.

Sounds like you might have had a disk full condition at a very inappropriate time.

vinodlakshmanan · Post by **vinodlakshmanan** » Thu May 12, 2005 7:22 am

The file owner is dsadm. Do I need to log-in to uv using dsadm, how is that possible? uv logs on using the Project Name AFAIK. I tried using Administrator using dsadm login and still I do not have permissions. Which admin is being referred to here?

ArndW · Post by **ArndW** » Thu May 12, 2005 8:18 am

Vinod,

make sure that the directory and the contents (mainly DATA.30 and OVER.30) are owned by dsadm and that the owner has '7' access (you could always do a chmod -R 777 DS_JOBOBJECTS plus the same for
D_DS_JOBOBJECTS and I_DS_JOBOBJECTS. Then, when you are logged in to UNIX as dsadm and attached to this directory (with the `cat /.uvhome`/dsenv environment loaded) enter the command "dssh"; this should bring you to the ">" prompt. From there enter your UVFIXFILE command; you exit back to your unix shell with the QUIT command.

Do you have a project UNIX level backup or a complete EXPORT?

vinodlakshmanan · Post by **vinodlakshmanan** » Thu May 12, 2005 8:23 am

Both actually. This project was being migrated from DS 6 to DS 7 and hence the project DataStage export is available. I plan to take a UNIX level backup too. But why is this needed, we are just fixing a hash file? What can UVFIXFILE do that can damage the entire project?

ArndW · Post by **ArndW** » Thu May 12, 2005 8:43 am

Vinod,

right now your project is unuseable right? So the UVFIXFILE will not damage anything, it will repair the file with likely data loss so that you will at least be able to save something. If you have a very recent UNIX backup, then restore the DS_JOBOBJECTS and I_DS_JOBOBJECTS back over the existing (corrupt ones) and your problem will be corrected.

Your design-time information is found in DS_JOBOBJECTS, so missing entries in this file will make job(s) and sequence(s) unusable. It is a critical file.

Your first step is to make the file accessible again, and then you will need to decide how to repopulate it.

battaliou · Post by **battaliou** » Thu May 12, 2005 8:54 am

Bad forward link found

This is an overflow error probably caused by a bad sector on your disk. I'm doubtfull if you can recover from this.

If you have a recent export, I'd consider abandoning this project & creating a new one. Looks like your hard drive is about to fail some time soon.

ArndW · Post by **ArndW** » Thu May 12, 2005 9:02 am

Battaliou - the error message is not a hardware level (i.e. disk sector) error; it is a forward link in the Hash File linked list structure.

vinodlakshmanan · Post by **vinodlakshmanan** » Thu May 12, 2005 9:10 am

1. restore the DS_JOBOBJECTS and I_DS_JOBOBJECTS back over the existing (corrupt ones) and your problem will be corrected.
2. Your design-time information is found in DS_JOBOBJECTS, so missing entries in this file will make job(s) and sequence(s) unusable. It is a critical file.

Since it is a repository file, can we not use the DS_JOBOBJECTS file of some other project, clear the file and re-index to re-establish all the links again? Or is there some other information which cannot be got by re-indexing in DS_JOBOBJECTS?

battaliou · Post by **battaliou** » Thu May 12, 2005 9:15 am

Arnd - Ja, you're right, its been a long time since I've seen blink and flink errors. What do you think about creating a new project and abandoning this one?

ArndW · Post by **ArndW** » Thu May 12, 2005 9:17 am

The contents of each project's DS_JOBOBJECTS will depend upon the jobs and objects you have defined in that project. If the other project contains a duplicate of this one (i.e. if you have a DEV and a PRODuction project) then you can do this, otherwise you will be loading junk.

The re-indexing does nothing except to rebuild defined alternate keys from records in the file, so if that is empty you will just be creating empty indexes. If you go to all your projects and do a COUNT DS_JOBOBJECTS and get identical results then go ahead and try your method instead of progressing as I suggested.

If you really wish to solve the problem by loading DS_JOBOBJECTS from another project I will cannot stop you; but please make a backup before doing so.

vinodlakshmanan · Post by **vinodlakshmanan** » Thu May 12, 2005 9:26 am

No, I am not going to take the files from another project. Just asked. But, as I said earlier, it is a project we are porting from 6x to 7x, hence can I use the files from 6x? If yes, will taking DS_JOBOBJECTS and the DICT file only suffice?
Also, another generic query: how do we transfer hash files between servers - here 6x and 7x are on different servers. I have done it once before by tarring and ftping, but it does not seem the right way. Is there a better method?

ArndW · Post by **ArndW** » Thu May 12, 2005 9:51 am

Vinod,

I don't know if the DS_JOBOBJECTS have changed between 6 and 7 - if you do a "LIST DICT DS_JOBOBJECTS" in both projects and they are identical then you can overwrite the files at UNIX level.

Taking HASH files from one system to another can be done at UNIX but if they have I_... indices then you will have problems, as the index is linked to from the main file by an absolute pathname, so you will need to re-index them after copying them or moving them to another directory.

Have you finished your FIXFILE?

DSXchange

Urgent: Project has been corrupted

Urgent: Project has been corrupted

backups