DSXchange

None of which negates anything I posted. It appears that other circumstances - as well as reference links - can generate this warning. It's still just an alert that something unusual, or non-standard, exists in the design. It sounds like Ray is saying choosing a partitioning method other then Entir...

We remove datasets using ORCHADMIN command in unix. But once i removed a dataset using normal RM command in Unix. This deleted only descriptor file not the original data. My question is, By mistake if we delete datasets using normal RM command (only descriptor file is deleted), can we still found th...

Entire partitioning uses shared memory on SMP environments/configurations. Other partitioning algorithms do not (because they do not need to share anything). If you choose a different partitioning algorithm than Entire (or Auto) for a reference input, then this warning occurs. It's not a "bad&...

Entire partitioning uses shared memory on SMP environments/configurations. Other partitioning algorithms do not (because they do not need to share anything). If you choose a different partitioning algorithm than Entire (or Auto) for a reference input, then this warning occurs. It's not a "bad&...

i am doing a join on input data from vitual dataset and an oracle table. The key is a single column of integer type. Right now i am hash partitioning and sorting both the inputs. Instead of Hash can i use Modulus. Developer guide says YES. But if i use modulus, does it ensure related records stay in...

Good question. If you would look in the advanced tab within each individual stage for DataStage, you would see several options, such as "Execution mode", "Combinability mode" and "Preserve partition". The first option tells that specific stage whether it should run in ...

When i selected Partition type of SAME for the transformer (transformer name is FtrGrpFields), this is the INFO i get. This is not a warning nor an error. But I want to know what is this " FtrGrpFileds:Input dataset 0 has a partitioning method other than entire specified; disabling memory shari...

join stage has two inputs. right now i am hash partitioning both the input data to join stage based on an integer column. (i want same value columns stay on same node). Can i do the same using Modulus partition. Will modulus also ensure that related records stay on same node. What is the difference ...

Simply because it does so much more work. C++ code is NOT generated for the whole job as you assert - all that is generated is an OSH script that is a one-to-one mapping of the stages in your design. Transformer and Build stages do have to compose, compile and link C++ code which is more time consu...

When i compile my job after making a small change in parallel transformer, it takes lot of time to compile. I know in the backend a c++ code is running for xformer,but still c++ code generates for whole job. You can say becuase of orchestrare transformer function on server xformer stage. But still i...

When i put same partition in xformer theres a warning saying only entire is allowed? Why is this warning coming? cant i put same partiton in transformer. And as well what partition i have to use for lookup reference link and input link. Should i use entire partition for reference links.

kumar_s wrote:Auto partiton could lead to RoundRobin partition is any stage and the same could be propagated. And thus the records could have been let on different nodes during duplicate removal process.

Does always partition always leads to roundrobin in any stage?

ray.wurlod wrote:Of course they do. They're doing the same processing.

What's a "performance compact"?

sorry its performance impact

Process data containing fewer distinct reference key values would be one way. Use more partitions would help too. Finding a different strategy, such as a sparse lookup or a Join or Merge stage might be another alternative. As you note, it is only an alert. You could create a message handler to demo...

From filter i am outputiting 2 links. one link has a condition and other is Reject link. For this reject link i can even write a condition. Right now without writing any condition in filter i am just collecting the rejected records. wil the partitions work on Reject Link. when i tried it it is worki...