Page 1 of 1

Partitioning sort and Sort Stage

Posted: Thu Jul 31, 2008 6:07 am
by ZDC
description :
I got a job abort with an err saying
"
xf_OpCnt,0: Fatal Error: Unable to initialize communication channel on SRVDA007. This is typically caused by a configuration problem. Examples of typical problems include:
1) The temporary directory, identified by $TMPDIR and/or the scratch disks in your ORCHESTRATE configuration, is located on a non-local filesystem (e. g. mounted over NFS).
2) The temporary directory is located on a filesystem with insufficient space.
"
Action
1> Cleared the scratchdisk though it was just 10 mb datafiles . Didn't clear any thing from the Temp dir had 7315 files and total data 10 mb. But still the job failed .
2>The Ds code was doing a hash partitioning and sorting in a redup stage . Changed that to Sort Stage and "same" partition for the redup stage. And it worked.

that made me think what is the difference in case of memory usage in sort stage and partitioning sort ? Assuming same keys and Hash partitioning and same server load.

And should I get this error on the first place when it is DS windows server and the scratchdisk folder size is virtually unlimited ?

looking forward to your replies.
&
Thank you for sharing your views ....

Re: Partitioning sort and Sort Stage

Posted: Sat Apr 23, 2011 2:06 pm
by D0n1117
I had the same problem with the same error with a lookup stage with 5 sequential files (4 as lookups). Plenty of scratchdisk, space was not an issue as the error indicates. All partitioning was set to Auto. I changed the partitioning to "Same" for the input and "Entire" for the lookups and now it works. Explicitly set your partitions on your input and see if that works.

Re: Partitioning sort and Sort Stage

Posted: Thu Apr 28, 2011 1:38 am
by Sampath
Your using sort stage so i will use tem are for operaction so you should check space and your using NFS mount point so and problem in connection check it.

Posted: Thu Apr 28, 2011 1:46 am
by ray.wurlod
Unless you specifically set the memory property in the Sort stage it uses exactly the same amount of memory as the input link sort (set by the environment variable APT_TSORT_STRESS_BLOCKSIZE). So one of your theories bites the dust!

Wherever possible always use local disk for scratch.

Re: Partitioning sort and Sort Stage

Posted: Thu Apr 28, 2011 1:46 am
by ray.wurlod
Sampath wrote:Your using sort stage so i will use tem are for operaction so you should check space and your using NFS mount point so and problem in connection check it.
I just don't understand this answer at all! Does anyone?

Posted: Thu Apr 28, 2011 2:24 am
by jwiles
I don't quite understand that answer either. Nor do I understand why D0n1117 has felt it necessary to try to answer a nearly three year old post, especially with an answer he copy/pasted to two other old threads! :roll:

Regards,