Question About Datastage extract

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
palak08
Participant
Posts: 75
Joined: Thu Nov 04, 2010 5:54 pm

Question About Datastage extract

Post by palak08 »

Hi All,

I am running a server job which has source as Oracle and target as oracle.
Our Oracle and Datastage are installed on separate servers.

I have one SQL query which perform SELECT in 40 seconds. It Selects 20 million records. Now when I do same thing on datastage (select only--> by putting @FALSE condition in transformer) then it takes 35 minutes.

My design looks like
OCI-----> Transformer(@FALSE)--->OCI

In performance stats I can see total record count on input link as 20 million.

My question is when I put @FALSE condition in datastage job and performs only extract part, then what is happing exactly.

Is datastage Extracting entire 20 million records from oracle server and bringing to datastage server?
OR
It is only Selecting on database server?

Any help will be very much appreciated.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

It's important to understand the topology here. All selects from sources are brought into the DataStage engine where transformations occur before being pushed to the target. So if each of those components are on different servers you could have a lot of information flying across the network that you need to be cognizant of.
-craig

"You can never have too many knives" -- Logan Nine Fingers
karthi_gana
Premium Member
Premium Member
Posts: 729
Joined: Tue Apr 28, 2009 10:49 pm

Post by karthi_gana »

where can i find more information on datastage engine architecture?
Karthik
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

In the manuals, particularly the Information Server Administration and Installation manuals.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply