Incremental Load

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
seeta
Participant
Posts: 8
Joined: Mon Nov 07, 2005 8:10 pm

Incremental Load

Post by seeta »

Hi
I want to load more records for the same job next day what I want to do to increase the records for next run.
Is there any way to increase the load for next run

Thanks
kwwilliams
Participant
Posts: 437
Joined: Fri Oct 21, 2005 10:00 pm

Re: Incremental Load

Post by kwwilliams »

More information will help you get a better answer. What are you loading from, a file stage, an Oracle stage? What are you loading to and what would you like the criteria to be for increasing the number of records loaded (date, key ) ?
nivas
Participant
Posts: 117
Joined: Sun Mar 21, 2004 4:40 pm

Re: Incremental Load

Post by nivas »

Hi

The answer simply is pass more records. Please post topic with more clarity.

thanks
Nivas
kwwilliams wrote:I want to do to increase the records for next run.
Is there any way to increase the load for next run
seeta
Participant
Posts: 8
Joined: Mon Nov 07, 2005 8:10 pm

Post by seeta »

I am loding from seqential file stage to oracle database,I am loading data comming from source.
I want to increase the load data by date
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I want to increase the load data by dateseeta,

your question is still unintelligible. We understand you are loading from a sequeuntial file to an oracle database. But what does
I want to increase the load data by date
mean in this context? You want to increate the value of a date column? You want to add data every day (and what is stopping you?)?
ameyvaidya
Charter Member
Charter Member
Posts: 166
Joined: Wed Mar 16, 2005 6:52 am
Location: Mumbai, India

Post by ameyvaidya »

From what i could figure out from the subject/replies...

Requirement:

How do I do Incremental loads?

Source:Sequential file
Target: Oracle database.
Update frequency: daily

Questions that need to be answered:

How is the sequential file created? Is a new file every cycle containing the changed/updated records or is it like a complete dump of the source?

How will the update timestamp be provided in the source sequential file? Is it a part of the header or as a column for every record?

Possible helpful links:

DsXchange Search results:
<a href="viewtopic.php?t=93060&highlight=increme ... ">First</a>
<a href="viewtopic.php?t=96868&highlight=increme ... >Second</a>
<a href="viewtopic.php?t=86234&highlight=increme ... ">Third</a>

<a href="http://blogs.ittoolbox.com/bi/websphere ... p">Vincent McBurney's Blog: Incremental Loads save the day</a>
<a href="http://wiki.ittoolbox.com/index.php/HOW ... b">Vincent McBurney's Howto DataStage: perform change data capture in an ETL job</a>
Amey Vaidya<i>
I am rarely happier than when spending an entire day programming my computer to perform automatically a task that it would otherwise take me a good ten seconds to do by hand.</i>
<i>- Douglas Adams</i>
kwwilliams
Participant
Posts: 437
Joined: Fri Oct 21, 2005 10:00 pm

Post by kwwilliams »

Seeta asked the question. I just in trying to get more data out of him about his problem made some suggestions about the kind of informatino that would be helpful. Right now I know Sequential file is the source and that is all.

Sorry if I created confusion in trying to get more information. Original question are subsequent post from SEETA are the only ones that provide any insight into his problem.
Post Reply