Error in preserve partitioning when Joiner Stage Used

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
sushanth
Participant
Posts: 18
Joined: Sat Oct 09, 2004 10:13 am
Location: Malaysia
Contact:

Error in preserve partitioning when Joiner Stage Used

Post by sushanth »

Two Datasets are joining into a Target DB2 table with left outer Join
In all the stages, Iam making Sequential mode enabled then I receive all records which Iwant in the target
but with a warning like "When checking operator: Sequential operator cannot preserve partitioning of parallel dataset on input port 0 "

If I made all modes of all stages into Default Parallel, I will not receive warning but one record from right table will miss
pls do describe
T42
Participant
Posts: 499
Joined: Thu Nov 11, 2004 6:45 pm

Post by T42 »

Do not worry about this warning message if this solution is producing the right data for you.

Normally, the Join Stage already handle the sort/partitioning for you, but in your observed case, it does not do the job. Maybe the datasets are sorted/partitioned earlier (and this noted behavior do propagate). You are overriding this behavior.

Do check earlier in the stream to see if you are doing sort/partitioning where you really don't have to. Otherwise, as long as you confirm with a large scale random test data (I'm talking about 1,000 rows per node), do not worry about this warning message.
sushanth
Participant
Posts: 18
Joined: Sat Oct 09, 2004 10:13 am
Location: Malaysia
Contact:

Error in preserve partitioning when Joiner Stage Used

Post by sushanth »

Thanks, then we are not doing any sorting or Partitioing at the earlier stream means preceding stage (?) still we get this error.
T42
Participant
Posts: 499
Joined: Thu Nov 11, 2004 6:45 pm

Post by T42 »

The framework actually insert in partitioning tasks where it is necessary/set by default. Read the online help for each stage, and use $APT_DUMP_SCORE to oserve this behavior. The information shared is a bit hard to read, though.
Post Reply