Partitioning sort and Sort Stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ZDC
Premium Member
Premium Member
Posts: 15
Joined: Fri Nov 23, 2007 5:06 am

Partitioning sort and Sort Stage

Post by ZDC »

description :
I got a job abort with an err saying
"
xf_OpCnt,0: Fatal Error: Unable to initialize communication channel on SRVDA007. This is typically caused by a configuration problem. Examples of typical problems include:
1) The temporary directory, identified by $TMPDIR and/or the scratch disks in your ORCHESTRATE configuration, is located on a non-local filesystem (e. g. mounted over NFS).
2) The temporary directory is located on a filesystem with insufficient space.
"
Action
1> Cleared the scratchdisk though it was just 10 mb datafiles . Didn't clear any thing from the Temp dir had 7315 files and total data 10 mb. But still the job failed .
2>The Ds code was doing a hash partitioning and sorting in a redup stage . Changed that to Sort Stage and "same" partition for the redup stage. And it worked.

that made me think what is the difference in case of memory usage in sort stage and partitioning sort ? Assuming same keys and Hash partitioning and same server load.

And should I get this error on the first place when it is DS windows server and the scratchdisk folder size is virtually unlimited ?

looking forward to your replies.
&
Thank you for sharing your views ....
Zero Degree Consulting
D0n1117
Premium Member
Premium Member
Posts: 11
Joined: Sun Dec 19, 2010 1:49 pm
Location: VA

Re: Partitioning sort and Sort Stage

Post by D0n1117 »

I had the same problem with the same error with a lookup stage with 5 sequential files (4 as lookups). Plenty of scratchdisk, space was not an issue as the error indicates. All partitioning was set to Auto. I changed the partitioning to "Same" for the input and "Entire" for the lookups and now it works. Explicitly set your partitions on your input and see if that works.
Don
DataStage Developer
Sampath
Participant
Posts: 21
Joined: Tue Apr 05, 2005 2:03 am

Re: Partitioning sort and Sort Stage

Post by Sampath »

Your using sort stage so i will use tem are for operaction so you should check space and your using NFS mount point so and problem in connection check it.
Regards,
Sampath
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Unless you specifically set the memory property in the Sort stage it uses exactly the same amount of memory as the input link sort (set by the environment variable APT_TSORT_STRESS_BLOCKSIZE). So one of your theories bites the dust!

Wherever possible always use local disk for scratch.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Re: Partitioning sort and Sort Stage

Post by ray.wurlod »

Sampath wrote:Your using sort stage so i will use tem are for operaction so you should check space and your using NFS mount point so and problem in connection check it.
I just don't understand this answer at all! Does anyone?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Post by jwiles »

I don't quite understand that answer either. Nor do I understand why D0n1117 has felt it necessary to try to answer a nearly three year old post, especially with an answer he copy/pasted to two other old threads! :roll:

Regards,
- james wiles


All generalizations are false, including this one - Mark Twain.
Post Reply