Failure trying to view Dataset

sjordery · Post by **sjordery** » Thu Jun 05, 2008 5:16 am

Hi All,

I have searched extensively for this, as I thought it must have come up before, but couldn't find it, so here goes...

I have a job that runs a number of times, parsing XML and appending to a dataset as output. That dataset is then opened up in a downstream job to write to a DB. The parsing jobs all finish fine, reporting the right number of output rows and completing with no exceptions. The downstream job, however, fails - reports:

dsIdDataToLoad,0: Failure during execution of operator logic.

APT_CombinedOperatorController,0: Fatal Error: I/O subsystem: Open of /myDatasetsDirectory/IdMasterDataFull.ds.sjordery.ukdstaged1.0000.0000.0000.25bb.c9e1f779.0000.01ac3e1f failed: No such file or directory

node_node1: Player 1 terminated unexpectedly.

main_program: Unexpected exit status 1

main_program: Step execution finished with status = FAILED.

If I go into the parsing job and View Data on the output dataset, this also fails with:

>##F TFIO 000161 12:01:25(003) <APT_CombinedOperatorController,0> Fatal Error: I/O subsystem: Open of /data/Datastage/Projects/GEMXREF/Datasets/IdMasterDataFull.ds.sjordery.ukdstaged1.0000.0000.0000.25bb.c9e1f779.0000.01ac3e1f failed: No such file or directory
>##E TFPM 000192 12:01:25(000) <node_node1> Player 1 terminated unexpectedly.
>##E TFPM 000338 12:01:25(013) <main_program> Unexpected exit status 1
>##E TFSR 000011 12:01:30(000) <main_program> Step execution finished with status = FAILED.
##I TCOS 000026 12:01:30(001) <main_program> Startup time, 0:05; production run time, 0:00.

In the Datasets directory, the following files exist:

-rw-rw-r-- 1 sjordery users 655360 2008-06-05 11:57 IdMasterDataFull.ds.sjordery.ukdstaged1.0000.0000.0000.b4f.ca1111ab.0000.7d61f62a
-rw-rw-r-- 1 sjordery users 655360 2008-06-05 11:57 IdMasterDataFull.ds.sjordery.ukdstaged1.0001.0000.0000.bd8.ca1111b1.0000.6b61882a
-rw-rw-r-- 1 sjordery users 655360 2008-06-05 11:58 IdMasterDataFull.ds.sjordery.ukdstaged1.0002.0000.0000.c61.ca1111b7.0000.0949b842
-rw-rw-r-- 1 sjordery users 131072 2008-06-05 11:58 IdMasterDataFull.ds.sjordery.ukdstaged1.0003.0000.0000.cea.ca1111bd.0000.d149f042
-rw-rw-r-- 1 sjordery users 1703936 2008-06-05 12:01 IdMasterDataFull.ds.sjordery.ukdstaged1.0017.0000.0000.e03.ca111273.0000.9f9f0a6c

So, the view data is trying to view a specific file that does not exist.

I have tried doing orchadmin rm on the IdMasterDataFull.ds file, but this fails with:

##I TCOA 000024 12:14:50(001) <main_program> Deleting IdMasterDataFull.ds.
##I TFSC 000001 12:14:50(002) <main_program> APT configuration file: /tmp/aptoa37251e457046
##E TFDM 000076 12:14:50(003) <main_program> Unable to open descriptor file to update: Permission denied
##E TFDM 000000 12:14:50(004) <main_program> Error updating ORCHESTRATE File Dataset descriptor segment information for 'IdMasterDataFull.ds'.
##E TFDM 000000 12:14:50(005) <delete IdMasterDataFull.ds> Error while truncating data set "IdMasterDataFull.ds".
filesystem related error
##I TCOA 000026 12:14:50(006) <main_program> Delete FAILED for IdMasterDataFull.ds.

Any ideas much appreciated.

Thanks
S

ArndW · Post by **ArndW** » Thu Jun 05, 2008 5:56 am

Your dataset is broken, someone has deleted one or more of your data files. When you do a "orchadmin -f IdMasgterDataFull.ds" are any of the files listed actually present in the directory?

The easiest solution is to do a UNIX delete of the .ds file as well as removing all the IdMasterDataFul.ds.sjordery* files and starting afresh.

sjordery · Post by **sjordery** » Thu Jun 05, 2008 7:06 am

A matter of user error I'm afraid. I had one of the directory parms set up incorrectly in the jobs - so it was looking for the .ds header/pointer file in the wrong directory. There was a IdMasterDataFull.ds in the 'incorrect' directory, but a previous clean-up had removed the data files.

Apologies for wasting your time. More coffee required.

Cheers,
S