Controller problem

anupam · Post by **anupam** » Thu Jun 10, 2004 3:12 am

Hello,

I am also working with Arun Verma. We have executed this command while running the application but found at any point of time the files opened does not exceed 200 (approx).

As mentioned earlier, the value in uvconfig for T30FILE is
"T30FILE 1000"

Please suggest is there any other parameters also which needs to be set. We have 24 CPU M/c 48 GB RAM. I don't think that there is any resource relatad issues. But there might be some setting at the Datastage Configuration level which needs to be rectified.

For Reference I am listing the various values in uvconfig

MFILES 50
T30FILE 1000
OPENCHK 1
WIDE0 3dc00000
UVSPOOL /tmp
UVTEMP /tmp
SCRMIN 3
SCRMAX 5
SCRSIZE 512
QDEPTH 16
HISTSTK 99
QSRUNSZ 2000
QSBRNCH 4
QSDEPTH 8
QSMXKEY 32
TXMODE 0
LOGBLSZ 512
LOGBLNUM 8
LOGSYCNT 0
LOGSYINT 0
TXMEM 32
OPTMEM 64
SELBUF 4
ULIMIT 128000
FSEMNUM 23
GSEMNUM 97
PSEMNUM 64
FLTABSZ 11
GLTABSZ 75
RLTABSZ 75
RLOWNER 300
PAKTIME 300
NETTIME 5
QBREAK 1
VDIVDEF 1
UVSYNC 1
BLKMAX 8192
PICKNULL 0
SYNCALOC 0
MAXRLOCK 74
ISOMODE 1
PKRJUST 0
PROCACMD 0
PROCRCMD 0
PROCPRMT 0
ALLOWNFS 1
CSHDISPATCH /usr/bin/csh
SHDISPATCH /usr/bin/sh
DOSDISPATCH NOT_SUPPORTED
LAYERSEL 0
OCVDATE 0
MODFPTRS 1
THDR512 0
UDRMODE 0
UDRBLKS 10
MAXERRLOGENT 100
JOINBUF 4095
64BIT_FILES 0
TSTIMEOUT 60
PIOPENDEFAULT 0
MAXKEYSIZE 768
SMISDATA 0
EXACTNUMERIC 15
MALLOCTRACING 0
CENTURYPIVOT 1930
SPINTRIES 0
SPINSLEEP 0
DISKCACHE -1
DCBLOCKSIZE 16
DCMODULUS 256
DCMAXPCT 80
DCFLUSHPCT 80
DCCATALOGPCT 50
DCWRITEDAEMON 0
RECACHESIZE 32
DMEMOFF 0x137f4000
PMEMOFF 0x1433c000
CMEMOFF 0x14340000
NMEMOFF 0x1a2a2000

Please suggest if some parameter needs to be modified.

ray.wurlod · Post by **ray.wurlod** » Thu Jun 10, 2004 6:31 pm

Having eliminated T30FILE as a contender, we now need to investigate whether there's really a problem with the status file RT_STATUS1885.

Open a TCL prompt or Administrator client window into the project, and issue a query on it, for example COUNT RT_STATUS1886 or (if you prefer SQL) SELECT COUNT(*) FROM RT_STATUS1886; and let us know the result.

Just to confirm that this is the correct status file, can you please verify using the command SELECT NAME, CATEGORY, JOBNO FROM DS_JOBS WHERE NAME = 'J3BlaSrc'; that JOBNO is indeed 1886?

Once you've posted these results we can use far less guesswork as to the next step. Basically if the RT_STATUS1886 table is corrupted, we need either to repair or replace it. If it isn't, we need to seek some other cause. If a query (such as those above) that processes the entire table succeeds without error, then the table is not corrupted.

anupam · Post by **anupam** » Thu Jun 10, 2004 11:32 pm

ray.wurlod wrote:
1. SELECT COUNT(*) FROM RT_STATUS1886;

2. SELECT NAME, CATEGORY, JOBNO FROM DS_JOBS WHERE NAME = 'J3BlaSrc';

Hello Ray,

After verifying the Job No by using [2] statement, which is 1886 only, i executed the [1] sql statement and the result i am getting is 24. As the sql statement is executed without any error which implies that the table is not corrupted.

ray.wurlod · Post by **ray.wurlod** » Fri Jun 11, 2004 6:56 pm

To summarise thus far, your controlling job generated the error

J0BlaSrc..JobControl (@J3BlaSrc): Controller problem: Error calling DSAttachJob(J3BlaSrc)
(DSGetJobInfo) Failed to open RT_STATUS1886 file.
(DSOpenJob) Cannot open job J3BlaSrc - not a runnable job

We have eliminated the open dynamic hashed file table (T30FILE) as a cause. We have eliminated a corrupted RT_STATUS1886 file as a cause.

This error occurred in the job control code in J0BlaSrc.JobControl job when it tried to use the DSAttachJob, then again when it tried to call DSGetJobInfo with an invalid job handle.

What you need to do now is to insert some diagnostic statements into the J0BlaSrc.JobControl job. You can use $DEFINE to conditionally compile the diagnostic statements. New statements are shown in red. Let's start by seeing what the problem with DSAttachJob is.
$DEFINE TESTING
* When you've finished testing, change to $UNDEFINE
hJob = DSAttachJob("J3BlaSrc", DSJ.ERRNONE)
$IFDEF TESTING
Diag = "Value returned by DSAttachJob = " : hJob
Call DSLogInfo(Diag, " * Testing *")
JobName = DSGetJobInfo(hJob, DSJ.JOBNAME)
Diag = "Returned job name = " : Quote(JobName)
Call DSLogInfo(Diag, " * Testing *")
* If the DSAttachJob failed, this should be DSJE.BADHANDLE
$ENDIF

From what I know of your site, I'm surprised to see a J3-level job being started from a J0-level job control routine. Ought there not to be a J1-level and a J2-level control routine involved?

Also, when you say parallel execution, are you running separate jobs in parallel, and/or are you using multiple-instance execution (which was planned, but not implemented, when I was there).

tonystark622 · Post by **tonystark622** » Mon Jun 14, 2004 9:17 am

Arun,

I have had similar problems when my jobs were run by more than one user and I didn't have DataStage set up properly. I had to make sure that I had umask 002 at the top of the dsenv file, and the group permission on the project directory was set to rws (note the s instead of x). Of course all users that run this job should belong to the dstage group (or whatever group you've used in place of dstage.

I hope this helps,
Tony

mhester · Post by **mhester** » Mon Jun 14, 2004 9:44 am

Anupam,

I have experienced this error on a couple of different occasions. I did not go into a lot of diagnostics, rather I saved the job as {JOB}Save (or whatever), deleted the "production" copy of the job and then saved the {JOB}Save back as the original name, recompiled and the problem disappeard.

Give it a try - it just might work.

I know it does not solve the real issue, but this might get you up and running and will give you some breathing room to further investigate with Ascential or the forum.

Regards,

Michael Hester

arunverma · Post by **arunverma** » Sat Jun 19, 2004 1:01 am

Dear Ray,

Thanks for your valuable information. Can you please elloborate the suggestion provided as where to code this statement in the job. I am not clear that where do i incorporate this statement?

We have implemented multi instance job after you left. We are getting maximum of this erorr only in this multiinstance job.

Thanks once again

Arun

ray.wurlod · Post by **ray.wurlod** » Sat Jun 19, 2004 5:13 pm

Arun,

As you will see in the code, it is intended to determine whether the DSAttachJob function was called successfully. Therefore the code is inserted around the DSAttachJob function that is invoked to get a job handle used to run the job.

If you're using multi-instance jobs and using an invocation ID in DSAttachJob, you might also like to interrogate the attached job to determine what it actually believes its invocation ID might be, and how many other invocations are already running (refer to the help on DSGetJobInfo for the relevant constants).

For example, could you be running into your DataStage licence limit? From memory (of course, I was there some time ago) you have an 8 CPU DataStage licence, even though your hardware has more CPUs than that.

Regards,
Ray