Error while deleting a job

kumar_s · Post by **kumar_s** » Wed Jan 10, 2007 4:42 pm

Are you using dsadm for all the time. Try running the job, where your RT_BP806.O will be modified. You can have another chance. What happens if you Export and import the same file (without any rename). It should overwrite all these file, provided if it has permission.

kumar_s · Post by **kumar_s** » Wed Jan 10, 2007 4:43 pm

Btw; What is the version of your DS. Where the DS.TOOLS doesn't have Check Integrity of Job files.

ray.wurlod · Post by **ray.wurlod** » Wed Jan 10, 2007 4:56 pm

You can't delete things that are in use (for example open in Designer or Directory Log View).

roblew · Post by **roblew** » Wed Jan 10, 2007 5:40 pm

I've already tried to import the same job to attempt to overwrite the job. It results in the error: Cannot open file RT_BP806.O

I've made sure I'm logged out of Designer and Director when trying to delete the job.

Yes, I'm using dsadm to delete. BTW, some of the directories are owned by user "koky", but also owned by dstage group. The directories owned by dsadm were actually created by the DS.CHECKER command I ran earlier today. The only directory I created manually was RT_BP806.O. I received the same error before and after creating this RT_BP806.O directory. The directory was empty until I compiled the job.

$ ls -ld *806*
drwxrwsr-x 2 koky dstage 4096 Jan 10 10:15 DS_TEMP806
drwxr-sr-x 2 dsadm dstage 4096 Jan 10 13:06 RT_BP806
drwxrwxrwx 2 dsadm dstage 4096 Jan 10 13:23 RT_BP806.O
drwxr-sr-x 2 dsadm dstage 4096 Jan 10 13:06 RT_CONFIG806
drwxr-sr-x 2 dsadm dstage 4096 Jan 10 13:06 RT_LOG806
drwxrwsr-x 2 koky dstage 4096 Jan 10 13:18 RT_SC806
drwxr-sr-x 2 dsadm dstage 4096 Jan 10 13:06 RT_STATUS806
$ ls -l RT_BP806.O
total 124
-rwxrwxr-x 1 dsadm dstage 120840 Jan 10 13:13 V0S203_CopyOft_P1P_CV00122_KNVP_criteria_4mayyin_xfmTrim.so

We are on DS EE 7.5.1A. Yes, I know 7.5.2 is out. We are not able to upgrade just yet. I will probably move everything to a new project before then.

kumar_s · Post by **kumar_s** » Wed Jan 10, 2007 5:47 pm

Have you tried running the job using dsadm (If at all its possible)?

roblew · Post by **roblew** » Wed Jan 10, 2007 5:53 pm

Interesting... The job compiled ok. Then I tried running the job from Designer, but ran into the same ol' error: Cannot open file RT_BP806.O

Then I tried to look at the log in Director, and got this message:
Error selecting from log file RT_LOG806
Command was: SSELECT RT_LOG806 WITH @ID LIKE '1N0N' COUNT.SUP
Error was: Unable to open DICT "RT_LOG806".

ray.wurlod · Post by **ray.wurlod** » Wed Jan 10, 2007 7:06 pm

Why are the setgid bits set in your permissions? Who did this, and why?

roblew · Post by **roblew** » Wed Jan 10, 2007 7:12 pm

this is because our developer's unix id's have their own group as their primary group (instead of dstage). Therefore, we set the guid on the Projects directories recursively and intentionally, so that all of these files will be owned by the dstage group. Otherwise, developers will not be able to edit each others jobs.

This was only done because we did not want to set dstage as the primary group for our users, since our unix id's are used for many other things (other than DataStage). And also since the users are all kept in NIS, the id's are not local to the DataStage box.

Does that make sense? I know this has caused us a problem in the past when using the SAP BW PACK stages, resulting in some SIGSEGV errors. But other than that, it seems to work as expected... I wouldn't think this is related to my problem with deleting this job, because I can delete other jobs just fine. This is the first time I've encountered it.

ray.wurlod · Post by **ray.wurlod** » Wed Jan 10, 2007 7:49 pm

It's probably unrelated, and might be worth a separate thread or even an FAQ on approaches to securing DataStage projects. Usually you have a separate group per project, and that's the group associated with objects in and under the project. The dstage group (which doesn't have to be called that) gives everyone access to the executables, global structures, and so on. And you set umask for all developers to 002 so that each can edit others' work in the same project.

You probably need to check the structural integrity of the hashed files that are giving the problems. Execute these commands.

Code: Select all

LIST.ITEM VOC "RT_LOG806"
LIST.DICT RT_LOG806
LIST.DICT RT_LOG
SELECT COUNT(@ID) FROM RT_LOG806;

Let us know the results.

roblew · Post by **roblew** » Thu Jan 11, 2007 11:58 am

We do have groups for each project, which are used to secure each project's data files, though the project files are owned by dstage. We also use project-specific groups for controlling access to the projects. But I think the problem is that since the primary group is not dstage or the project specific group, umask of 002 still wouldn't allow others to edit each other's jobs (since files will be created owned by that user, AND by that user's primary group.

Code: Select all

$ id user1234
uid=25798(user1234) gid=25798(user1234) groups=25798(user1234),25706(olympic),25907(dstage),25705(olympic_dsdev)

file created with guid on parent directory:
-rw-rw-r--    1 user1234     olympic      4738 Jan  2 16:44 test.ds
file created without guid:
-rw-rw-r--    1 user1234     user1234      4738 Jan  2 16:44 test.ds

job directories created with guid on parent directory:
$ ls -ld *7989*
drwxrwsr-x    2 cgey     dstage       4096 Dec 14 19:08 DS_TEMP7989
drwxrwsr-x    2 cgey     dstage       4096 Dec 14 19:08 RT_BP7989
drwxrwsr-x    2 cgey     dstage       4096 Dec 14 19:10 RT_BP7989.O
drwxrwsr-x    2 cgey     dstage       4096 Dec 14 19:08 RT_CONFIG7989
drwxrwsr-x    2 cgey     dstage       4096 Dec 14 19:08 RT_LOG7989
drwxrwsr-x    2 cgey     dstage       4096 Dec 15 02:32 RT_SC7989
drwxrwsr-x    2 cgey     dstage       4096 Dec 14 19:08 RT_STATUS7989
job directories created with guid on parent directory:
$ ls -ld *7989*
drwxrwsr-x    2 cgey     cgey       4096 Dec 14 19:08 DS_TEMP7989
drwxrwsr-x    2 cgey     cgey       4096 Dec 14 19:08 RT_BP7989
drwxrwsr-x    2 cgey     cgey       4096 Dec 14 19:10 RT_BP7989.O
drwxrwsr-x    2 cgey     cgey       4096 Dec 14 19:08 RT_CONFIG7989
drwxrwsr-x    2 cgey     cgey       4096 Dec 14 19:08 RT_LOG7989
drwxrwsr-x    2 cgey     cgey       4096 Dec 15 02:32 RT_SC7989
drwxrwsr-x    2 cgey     cgey       4096 Dec 14 19:08 RT_STATUS7989

Yes, we could change the group owner of the project directories to olympic instead of dstage to be even more secure, but we would still need the guid set. I agree that this topic could/should be its own thread.

Anyhow, here's the output of the commands you listed (I'm guessing this is not what you expected?):

Code: Select all

>LIST.ITEM VOC "RT_LOG806" 09:26:55  01-11-07  PAGE    1

    RT_LOG806
001 F  Ardent DataStage Job File
002 RT_LOG806
>LIST.DICT RT_LOG806
Unable to open "DICT RT_LOG806" file.
DICT RT_LOG    09:28:12  01-11-07  Page    1

               Type &
Field......... Field. Field........ Conversion.. Column......... Output Depth &
Name.......... Number Definition... Code........ Heading........ Format Assoc..

@ID            D    0                            RT_LOG          10'0'R S
TYPE           D    1                            Type            2R     S
WAVE.NO        D    2                            Wave No.        8R     S
TIMESTAMP      D    3                            Timestamp       19L    S
MSG.ARGS       D    5                            Message Args    15L    M
MSG.TEXT       D   10                            Message Text    60T    M
FULL.TEXT      I      SUBR("*DataSt              Full Text       60T    M
                      age*DSR_MESSA
                      GE",@ID,MSG.T
                      EXT,RAISE(MSG
                      .ARGS))
SEVERITY       I      FIELD("Info,W              Severity        8L     S
                      arning,Fatal,
                      Reject,Starte
                      d,Reset,Error
                      ,Debug",",",T
                      YPE)
@              PH     ID.SUP
                      SEVERITY
                      FULL.TEXT
                      WITH @ID LIKE
                      '0N' BY @ID

9 records listed.
>SELECT COUNT(@ID) FROM RT_LOG806;
Unable to open "" file.
>SELECT COUNT(@ID) FROM RT_LOG806;
Unable to open "" file.

[/code]

ray.wurlod · Post by **ray.wurlod** » Thu Jan 11, 2007 2:38 pm

For some reason the VOC entry does not include the pathname of the file dictionary for job log. Take a look at any of the other job log entries (maybe RT_LOG805) to see what I mean. The fix is

Code: Select all

UPDATE VOC SET F3 = 'D_RT_LOG' WHERE F0 = 'RT_LOG806';