combinality mode

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
legendkiller
Participant
Posts: 60
Joined: Sun Nov 21, 2004 2:24 am

combinality mode

Post by legendkiller »

What is the functionality of combinality mode in Advance tab of stage? Where it can be useful?
Kirtikumar
Participant
Posts: 437
Joined: Fri Oct 15, 2004 6:13 am
Location: Pune, India

Post by Kirtikumar »

DS can combine multiple stages as a single process. You can use this mode if you want to decide the combining by your own.

The mode can have values: Auto, Combinable, Don't Combine.
Auto: let the DataStage decide whether to combine the stages
Combinable: Combine the operator with other stage if possible
Don't Combine: do not combine them at all

As multiple stages are running as a single process, sync overhead in stages is also minimized.
Regards,
S. Kirtikumar.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Hi,
May i know some of scenario in which we can use the diferent options.

regards
kumar
DEVESHASTHANA
Participant
Posts: 47
Joined: Thu Sep 16, 2004 5:26 am
Location: India

Post by DEVESHASTHANA »

Kirti,

Can u explain combinability mode in more detail,
What do you mean by combine the operator with other stage if possible,
Please explain this with some example,

Thanking you in anticipation,

Regards,
Devesh
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Hi,
It is in the manuals,but...
combine means try to combine what ever you can to 1 operator.
don't combine means leave all operators seperate as in the job design.
if you read between the lines you see that the later will probably get you better performance.

IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

We usually leave it at the default combine mode for faster processing. The only time we change it is when a lookup and a transformer get combined and the reject behaviour becomes different to what we want. You can have a lookup stage with no reject link that sends rows down the transformer reject link before it reaches transformer processing.
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

The logic behind not combining operators being faster can be explained via a simple "fill the pool" example.
let us say you want to fill your pool with water.
you have 2 choices:
1. use a single long but narrow hose
2. put many shorter but a lot wider hoses 1 after the other.

suposing you can thruput to maximum any hose caliber, what choice will get your pool filled up faster?
(assuming were not talking 1 pint pools aka bear glases ;))
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
Post Reply