DS questions

mydsworld · Post by **mydsworld** » Fri Feb 09, 2007 1:18 am

Please let me know the following :

1. How can we perform a incremental data load in datastage
2. Can we restart a job sequence from a certain point on failure instead of running the whole sequence

I_Server_Whale · Post by **I_Server_Whale** » Fri Feb 09, 2007 1:24 am

Search the forum. They have been covered before. Looks like they are some interview questions.

chulett · Post by **chulett** » Fri Feb 09, 2007 8:27 am

Short answers:

1) Yes, any kind of load actually.

2) Yes, and the methodology is documented in the manuals.

As noted, you'll find many, many (many) conversations on those topics here.

michaeld · Post by **michaeld** » Sun Feb 11, 2007 10:35 am

>1. How can we perform a incremental data load in datastage

this is more of an ETL question and not a DS question. One method is to maintain a copy of the target table as a dataset and check for the difference with your input file every time the job starts. That will get you an incremental data set.

A lot of times it is faster to do this using the database so that you do not need to send a lot of data over the network. You can maintain incremental changes with triggers or do the difference of two table dumps (previous and current) on the database server. It depends on your system setup.

>2. Can we restart a job sequence from a certain point on failure instead of running the whole sequence

In your sequence job, you have to go to job properties and check the restartable option. This can also be set in Administrator as a project wide setting. A job will only restart from a checkpoint if it fails. Otherwise it will think that the run was successful. So you can either set it to "fail after 1 warning" in director or make it fail on a warning in the sequence job itselft.