Internal data error

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
attu
Participant
Posts: 225
Joined: Sat Oct 23, 2004 8:45 pm
Location: Texas

Internal data error

Post by attu »

After getting weird error messages
e.g

Code: Select all

Record \J\3185\VO on file DS_JOBOBJECTS cannot be wriiten..
I did count ds_jobobjects and count ds_jobs ( it gave me record counts)

but count ds_jobobjects gave me this internal data error

Code: Select all

File ' dsadm/Ascential/DataStage/Projects/myproject//DS_JOBOBJECTS/OVER.30
Computed Blink of 0x800does not match expected blink of 0xA5C000!
Detected within group starting at address 0x136F800!

Should I do a re-index all, before doing that I would like to mention that we had so many locks in this particualr project as well, i did the unlock all.
now i want to do the reindex all.

Any suggestions?

Thanks[/b]
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

You should call tech support. You've corrupted the main file that stores all jobs design information. You've probably lost some jobs, if not all.

Tech support will probably tell you the best thing to do is to get your most recent backup of the project directory and restore it. If you can't do that, you'll need to get a recent export of your project and re-import, but before you can do that you have to fix the DS_JOBOBJECTS file. That usually involves trying to repair it - with data loss. Worst case you'll have to clear it. No matter what, it's a bad situation.

You should also investigate how you corrupted it. Did you exceed 2.2GB's worth of job design information? It's a 32BIT hashed file so you're limited on size. Did your file system holding the project fill up?
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

It's not a 2GB problem; otherwise the BLINK address would have been 0x80000000.

The file may be repairable using fixtool; again, run this under advice from your support provider. Take a backup copy of DS_JOBOBJECTS before attempting any form of repair.

Even the repair tool may have to discard some data.

There are some experts in IBM (and elsewhere) who can repair hashed files manually, provided that the file is repairable. This involves manually resetting the pointers after calculating the correct values for them. They charge heavily for this service.

No, I do not intend to publish details here. You need deep knowledge of the internal structure of hashed files (and there are twelve variations of that), and that's just for starters. The ability to think in hexadecimal helps too.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
attu
Participant
Posts: 225
Joined: Sat Oct 23, 2004 8:45 pm
Location: Texas

Post by attu »

Hi kcbland,

We have opened up a ticket with IBM support.
here are the steps we will take:

Delete this project
Import this project from test server
I just checked for other projects, did a count ds_jobobjects, it returned records..I guess DS_jobobjectes file is working fine for other projects.
since we already got rid of the corrupted project so there is no harm in importing a new project from test server. right?
we are not sure what caused this corruption!!! atleast the project directory space did not cause this, we have 30 GB space available.
Thanks
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Too late to diagnose if you've already removed/recreated the project. As long as you can recover your jobs then you're fine. Best practice for DEV environments is to export your job design repository nightly so that you can always recover your work up to the previous end of day.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
attu
Participant
Posts: 225
Joined: Sat Oct 23, 2004 8:45 pm
Location: Texas

Post by attu »

Gentlemen,

Tell me how to fix the ds_jobobjects using fixtool, i never used it,
well after we deleted the corrupted project and imported a fresh one, we still faced some issues while doing an export by category, it threw the DataStage DSR_SELECT action=7 (30107) Error....was resolved after we did a re-index.

Is re-index the only solution to this issue, what is actually going haywire ?
I am really concerned about it!!!!
I appreciate the excellent information you both provided.

Thanks
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

If you've never used fixtool, involve your support provider. Take a backup copy of the hashed file first - fixtool can (in rare circumstances) damage the file further.

The fixtool executable is in $DSHOME/bin. You can use any or all options without -fix entirely safely to report what fixtool finds to be wrong with the file.

The actual repair is effected with the -fix option. Don't rush into this without seriously contemplating what's reported without this option.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
attu
Participant
Posts: 225
Joined: Sat Oct 23, 2004 8:45 pm
Location: Texas

Post by attu »

Ray,

For the backup, are you refering to the dynamic hashed file directory which has three files DATA.30, OVER.30 and .Type30.

Thanks for your help.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Yes. Simply take a copy of it. Copy the permissions as well.

Code: Select all

cp -irp hashedfiledir /otherdir/hashedfiledir
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
attu
Participant
Posts: 225
Joined: Sat Oct 23, 2004 8:45 pm
Location: Texas

Post by attu »

ray.wurlod wrote:Yes. Simply take a copy of it. Copy the permissions as well.

Code: Select all

cp -irp hashedfiledir /otherdir/hashedfiledir
...
Ray,

Which hashed file are we talking about here? Any directory starting with D or DS is a hashed file directory containing three files (DATA.30, OVER.30 and .Type30).
Can you please guide me a little bit, I want to handle it with care :?

Thanks
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The one that you intend to repair, in this case DS_JOBOBJECTS.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply