Perfomance Tuning in Change Capture Stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Suman
Participant
Posts: 53
Joined: Thu Oct 07, 2004 8:28 am

Perfomance Tuning in Change Capture Stage

Post by Suman »

Hi All,
I am using Change Capture stage for comparing previous and new records. The volume of records is 0.7 millions and the lenghth of a single record is 4750 char with some numeric fields in between.The job is taking 39-40 mins to complete. Also the records are sorted on both links with the change capture key. This sorting also is taking long time. Is there any environment variable to reduce the sort time or using separate sort stage is preferable.Please provide your valuable inputs.

Suman
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

How have you proven that it's the sort time?

I always prefer an explicit Sort stage, because it allows me to allocate more memory to the tsort operator. This can result in quicker finishing. It also allows sort mode of "don't sort (previously sorted)" where applicable.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

If it is previously sorted, you can as well include an environmental variable to avoid the auto sorting. APT_NO_SORT_INSERTION.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Post Reply