Reading Source Data by Key

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
Seyed
Participant
Posts: 74
Joined: Wed Apr 14, 2010 7:25 am
Location: Oklahoma City

Reading Source Data by Key

Post by Seyed »

Hi All,
I have a Server job that reads data from one Oracle table and loads another Oracle table. To do that I am using two Oracle OCI stages. I want to extract 11/15/2011 data from the source table. I put a constraint to achieve data. The problem is that when I run this job, the 11/14/2011 data is read first and ignored and then the 11/15/2011 data is loaded. The date column in the source table is indexed. Is there a way to skip the 11/14/2011 data and go directly to 11/15/2011 data? At this moment, the source table consists of data for 11/14/2011 through 11/18/2011 which means running this job the way it is would even taking longer when running for the 11/16/2011 through 11/18/2011. Each of the above 5 days have more than 3 million rows.

Thank you for your help,

Seyed
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

If you literally mean a constraint, that won't affect your Oracle data being sourced. You'd need to use the date in the where clause in order to leverage your index and only source the appropriate records.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Seyed
Participant
Posts: 74
Joined: Wed Apr 14, 2010 7:25 am
Location: Oklahoma City

Post by Seyed »

Hi Craig,
Thank you very much for the information. I am going to use the date in the where clause and let you know how it turns out.

Thanks again,

Seyed
Seyed
Participant
Posts: 74
Joined: Wed Apr 14, 2010 7:25 am
Location: Oklahoma City

Post by Seyed »

Hi Craig,
Using your suggestion, I was able to add the following 'where' clause to the source OCI stage and modify the date parameter [RUN_DAY] by defining it as string and setting it as '11/15/2011'.

Code: Select all

WHERE TO_CHAR(WEB_RAWDATA.LOG_DT_TM, 'MM/DD/YYYY') = #RUN_DAY#

Thank you very much,

Seyed
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

FYI - that way invalidates the use of the index over LOG_DT_TM, better to put a TO_DATE around your parameter value. Both will work, the change will make it work more better. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply