Sequence Performance Query

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Sequence Performance Query

Post by parag.s.27 »

I wanted to know about sequence performance.

Here I am having 100 plus small sequences that are having 4 - 5 jobs incorporated in each on an avg. Now these sub sequences are to be called in one master sequence which represents a Single Database.

What can be the best approach for moving forward: -
1.I should split this in 4 - 5 (sub master)sequences where in each of these I'll call 25 small subsequences. Later I'll incorporate these 4 - 5 sub master sequences in a single sequence.
2.I should directly go ahead and call all these 100 subsequences in a single sequence. But I think this will be more complicated and it will be more difficult to track and debug any error in it.
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

Modularizing and putting comments for log entries is for sure the best way. Option 2.
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
abhi989
Participant
Posts: 28
Joined: Mon Sep 19, 2005 2:31 pm

Post by abhi989 »

You will also need to take into consideration that if you are firing these many jobs at the same time, CPU might run out of processing power and starts crapping out. (need to take hardware into consideration). Also logging information into the job log can be done by calling a custom routine that does that(in this routin you can put a DS log command)
parag.s.27
Participant
Posts: 221
Joined: Fri Feb 17, 2006 3:38 am
Location: India
Contact:

Post by parag.s.27 »

abhi989 wrote:You will also need to take into consideration that if you are firing these many jobs at the same time, CPU might run out of processing power and starts crapping out. (need to take hardware into consideration). Also logging information into the job log can be done by calling a custom routine that does that(in this routin you can put a DS log command)
The Hardware is selected as per the requirements only...hence it is fully compatible.
Thanks & Regards
Parag Saundattikar
Certified for Infosphere DataStage v8.0
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

"Compatible" is an entirely different question from "enough". You can still overload compatible hardware.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply