warning in lookup stage

[bart12872]

Hi,

I have a Datastage job design, like that or something like that :

INPUT --> SORT+HASH PARTITION --> TRANSFORMER(partition : SAME, no sort) --> LOOKUP (partition : SAME, no sort) --> AGREGATION (partition SAME, no sort)--> OUTPUT

First, I partition and sort data, and I want to keep the partition and the sort all along the job.
So, I used SAME partitionning and no sort in all stages.
and for the reference lookup i use entire partitionning.

But,

Because I use same partitionning, I have the following warnings :
In the transformer :"Transformer_128: Input dataset 0 has a partitioning method other than entire specified; disabling memory sharing."
In the lookup : "Lookup: Input dataset 0 has a partitioning method other than entire specified; disabling memory sharing."

So, I read post in the forum, and for the transformer warning it's a Datastage bug due to the use of the lookup logic in transformer. This bug is fix in the 7.6 version -> ok
but, it seem that the lookup warnings is not a bug. So my questions (finally) are :

- Does this warning is a problem for the logic of my job ?
- What does "disabling memory sharing" mean ? what is the consequence for the performance ?
- perhaps it's also a bug because Datastage check for all lookup inputs to be in entire partionning while only reference input must be in entire, isn't it ?

thanks,
Martin.

[ray.wurlod]

It's not a bug. And there is no 7.6 version.

It's an alert. Only Entire partitioning can make use of shared memory for exchange of keys, or even needs to. All other partitioning algorithms don't need to share keys.

That you have chosen not to use Entire means that the use of shared memory for exchanging keys is disabled, and if your algorithm requires re-partitioning to occur then this will be done not via shared memory but via TCP sockets.

The alert message is warning you that such is the case.

[bart12872]

ok, so I try to use entire partitionning in input of the lookup and performance decrease, so I will keep the same partitionning and the warning.


ps : for the DS 7.6 version, the last post of this topic mentionned it (viewtopic.php?t=101149). Certainly, not PX.

[ray.wurlod]

There is no 7.6 version. The final 7 version is 7.5.3.