DSXchange

Hi,

I have 1,00,00,000 records of data.After loading 65,00,000 data the job has been aborted.

Is there any possibility to load data 65,00,001 onwards not starting onwards in datastage.

If it is possible Could you please suggest me??

No such checkpoint is there in datastage But you can always design a job for your requirement.

In other words, you need to design that capability into your jobs. How you do that depends on many factors such as the nature of both the load and the data being loaded. Primary question in my mind is - is your source data static? Will record #65000001 be the same on the rerun as it was on the first? Answering 'no' will obviously complicate things. Otherwise it could be as simple as a job parameter with a 'start record' counter that is the number of records to skip. Default to zero, override for a post-failure rerun.

It seems an Interview question and one of the common Interview question in India. Entire reload vs Check the loaded one and then start loading for the rest, which one is less time consumption will decide how to restart the job.

Answer to your question is YES.

But there is no straight forward option (Eg: Check box / ON - OFF) ask Datastage to start again from the record where it stopped!!

As Ray said there are ways and you can do that!

Craig, Ray... six of one, two dozen of another.

Let us assume that your primary Data having 1000 records and make a your target as a reference and by filtering the data based on any dummy value, then only records which are not already inserted will be loaded into target.

your job desing should be like this :


Reference(Target)
|
|
Join------>Filter----->Target
|
|
Source


suppose you had some key columns, write a User defined SQL
like Select 'Y' , KC1,KC2...etc., and Filter the Records in Filter stage Where COL='Y'.

Then the already inserted records will be Filtered out.