DSXchange

Hi everyone,

I searched through all the posted of mutex issues in this forum.
But it doesn't help to fix my jobs.
I have a job using IPC stage and getting this 'ds_ipcput() - timeout waiting for mutex'. I used default buffer size (128) and time out (10) previously and increased buffer size like 258 , 1000 and tested again.
Tried to increased the waiting time as well. But still have this error.

May I know how I can calculate buffer size and time out to overcome this?

Note: The job is running successfully with 1 million of records, but come to 2.3 million of records and then gave above problem.

Following is my job design:

ODBC --> IPC --> Agg -->Agg--> Seq file
|
Tranformer -->Agg--> Seq file


Thanks a lot,
sansan

sansan wrote:Hi everyone,

I have a job using IPC stage and getting this 'ds_ipcput() - timeout waiting for mutex'. I used default buffer size (128) and time out (10) previously and increased buffer size like 258 , 1000 and tested again.
Tried to increased the waiting time as well. But still have this error.

Note: The job is running successfully with 1 million of records, but come to 2.3 million of records and then gave above problem.

sansan

What Aggregation are you using?
If it is other than count(*) then, try the same with sorted data (you can use order by in other caluse), based on the columns which are grouped by in the Aggregator.

May be this will help you solve your problem.

Thanks for prompt reply.
1st agg, using first () function
2st agg and 3rd agg, using count(*).

Columns were grouped by ODBC stage are different from Agg stage and so I will try to use same columns used in agg stage to group by in ODBC stage.

sansan wrote: Columns were grouped by ODBC stage are different from Agg stage and so I will try to use same columns used in agg stage to group by in ODBC stage.

Could you please explain to me where the sort needs to be added? Is it in the Aggregator stage? Or in the Transformer?

I haven't used DataStage for a while and I'm very rusty.

The job has the following sequence:
Oracle Stage --> Transformer --> Aggregator --> Transformer --> ODBC

Thanks for your help.

Do the sort in your database stage itself by using order by cluase. Let the aggregator know what keys the incoming data is sorted on.

DSguru2B wrote:Do the sort in your database stage itself by using order by cluase. Let the aggregator know what keys the incoming data is sorted on.

Do you mean the first Oracle Stage? Do I need to add sort in the Select statement? Does it matter what sort I pick?

Yes, in your first OCI stage, in your select, you need to add 'order by' clause to sort the data. Its upto you to sort by as many columns as you want. The sort order must be the order of group by's in the aggregator.

Thanks for the quick reply. I'll try it.

DSguru2B wrote: Let the aggregator know what keys the incoming data is sorted on.

I need more help with some basics. In the Aggregator, under Inputs --> Columns, there are two columns Sort and Sort Order. Do I need to enter anything into Sort column? There is a dropdown box in the Sort Order column. The options are ignore, descending, ascending. No matter what I select, it dispappears as soon as I move to the next field/row. What am I doing incorrectly.

Thanks.

Pre sort is not really need for Server job aggregator. (Unless specified in sort option otherwise).
mutex - The lock held amog the process. Try to use IPC stage between two aggregator stage.

You need to supply the order of sort. Which depends upon your order by's.
For eg. if in your sql you have
order by col a, col b, col c
Then in your aggregator you need to specify for if you dont supply the sort num, the sort will dissappear no matter what you select from the drop down. For more info, read about the aggregator stage in your dsbooks.

kumar_s wrote:Try to use IPC stage between two aggregator stage.

Thanks for the suggestion. I'll look into IPC stage. However, I only have one Aggregator stage.
Here is my job sequence:
Oracle Stage --> Transformer --> Aggregator --> Transformer --> ODBC

I was looking at the first post.
Still you can use between both side of the Aggregator.

If you can do the sort as part of your database read, do it there! The database will be much, much faster then DataStage.

Then the sort in the aggregator, while it won't hurt, will be incidental, as suggested by Guru (or is that Chuck? )

Just make sure the order by clause in your SQL is the same as the grouping in your aggregator!

Rob

rwierdsm wrote:Then the sort in the aggregator, while it won't hurt, will be incidental.

Actually, asserting the 'sort' order in the Aggregator is required or else all your database sorting will be wasted.