shared container - increasing the speed

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
priyaram
Charter Member
Charter Member
Posts: 16
Joined: Fri Apr 23, 2004 7:39 pm

shared container - increasing the speed

Post by priyaram »

i created one shared container which contains one dynamic sql st , that calls the pl/sql procedure. and i called that shared container using a test job (i.e) parallel job.
while running the job in datastage it's taking twice the time as that of compared with pl/sql procedure on unix.
anybody having any idea about incresing the speed of datastage job
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

What stage type inside the shared container contains the "dynamic SQL" statement?

What you seem to be trying to accomplish here is to have the following architecture run at no cost; that is, to run at the same speed as if it weren't there. The layers are:
  • UNIX
    osh (several processes)
    DataStage job process(es)
    Shared container process(es)
    Stage type (ODBC?) executing "dynamic SQL"
    ODBC driver?
    Oracle client software
Every time you add a layer to your architecture, there's a cost. Indeed, if the shared container uses Server stages, there are even more layers.
You want to compare the above with:
  • UNIX
    Oracle client software (includes sqlplus)
Can't be done.
The obvious question is whether it's really necessary to invoke a stored procedure for every row. Could you do the same thing with a buildop, which would be much faster? Or re-think your design, so that the data involved resides in datasets while your job is running - therefore all data modifications are effected in memory.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
priyaram
Charter Member
Charter Member
Posts: 16
Joined: Fri Apr 23, 2004 7:39 pm

stage type

Post by priyaram »

inside the transfor mer stage , i am giving the dynamic sql statement
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

So, in DataStage you're generating the "dynamic SQL" for every row processed, while in PL/SQL (presumably) you're not. Is this the case? And you expect it to run at the same speed! Read my earlier post again.
I don't think you have much scope for improving performance without moving away from a "dynamic SQL" approach; this is not really how DataStage is designed to execute.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply