DSIPC_OPEN_TIMEOUT Error and Basic Transformer

[gsherry1]

Hello Forum,

I am executing a Parallel job containing a Basic Transformer. The transformer has an input from a funnel and 4 outputs (3 seq and one teradata enterprise stage). Basic Transformer is operating in parallel mode with a Don't Combine setting.

Job runs fine 80% of the time.

The other 20% of the time I receive sequences of errors such as the following:

Evaluate,0: Unable to open project 'DEV_DW00QL' - 81011.
Evaluate,1: Unable to run job - -2.
DW00QL_ExtractValidation.#1.XfmEvaluate.ToEvaluation-Input.ToEvaluation: ds_ipcgetnext() - timeout waiting for mutex

Or

DW00QL_ExtractValidation.#0.XfmEvaluate.ToEvaluation-Input.ToEvaluation: ds_ipcopen() - call to OpenFileMapping() failed - The system cannot find the file specified.
DW00QL_ExtractValidation(XfmEvaluate).#0.XfmEvaluate: |Error 11 in GCI Link initialisation.|

XfmEvaluate is my Basic Transformer.

According to documentation, the DSIPC_OPEN_TIMEOUT should be increased from the default of 30 should I experience problems using this stage. I have increased the value from 30 to 120, but this problem still pops up.

In every case when I get a failure, the startup time is greater than 30 seconds, but less than 40 seconds. Every case that the job succeeds, the startup time is between 20 and 30 seconds. This suggests to me that DS is ignoring my DSIPC_OPEN_TIMEOUT setting.

Your input is appreciated.

Thanks,

Greg

[gsherry1]

Enabling multiple instances seems to have eliminated this problem. Not certain why. Couldn't find any documentation that mentioned that running Basic Transformer in parallel requires this setting.

I do know that I don't have multiple executions of the job occuring at the same time.

[ray.wurlod]

I think that the multi-instance thing is a red herring, and has coincidentally occurred at a time when the system total load dropped.

The ipc errors are usually to do with row buffering. When you're using a BASIC Transformer the data have to be buffered and translated to and from typeless. This uses inter process buffering. When the system is heavily loaded you can get buffering timeouts.

[kumar_s]

Hi Greg,
Iam not sure, Still throw some light or Terradata as well.
timeout waiting for mutex my database people use to bother about this at times.

-Kumar

[ray.wurlod]

"Mutex" stands for "mutually exclusive" and is one mechanism by which semaphores are implemented. A semaphore is a place to wait. When a process waits on a mutex lock (because, for example, the lock is held by another process) it does not sleep but, instead, "spins its wheels" attempting to gain the lock. Code that invokes a "wait on mutex" must specify a maximum number of retry attempts ("spins") and a maximum time to wait. For DataStage these are configured by the uvconfig parameters SPINTRIES and SPINWAIT. Get advice from your support provider about appropriate values for your system before attempting to change these.

[gsherry1]

Hello Ray,

You are correct about being a red herring. I still received this error after specifying multiple instances, but received it much less frequently. I tried running the Basic Transformer in sequential mode, and it has not occured (yet).

I will try out your suggestions with SPINTRIES and SPINWAIT.

Thanks,

Greg