option while loading to database

[sdfasih]

Hi ,
I would like to know what update action I should select while loading data into data base when data already exist should not be deleted and we are running jobs parallely to load data into the database.

Is it update existing or insert new rows?

Thanx.

[kris007]

I don't think you can load data into the same table through different jobs at the sametime. However, the answer to your question is , if you don't delete the table and yet update the table you need to perform updates for the already existing rows and insert for new rows. You will have to define two seperate links , one for updates and one for inserts to achieve good performance. What is your target database? Using the Update existing or Insert new rows or viceversa takes a heavy toll on your performance and is not suggested unless your target table has very fews rows..as few as 100-200. IMO

[Krazykoolrohit]

make all jobs write to text files instead of directly to a database.

write a job to pick all these text files and remove duplicates and then load this file into the table.

[ArndW]

What a second here - you seem to be forgetting that databases are designed specifically to enable multiple concurrent accesses and have rules built in place to cater for parallel writes to the same record! You will need to use these rules in order to ensure a successful load and there is no reason to use text files for interim storage.

If you are loading data and do not wish to overwrite (or update) existing records then an "insert" will do the trick; albeit this is not the most efficient way as it generates a database I/O operation and will also generate a failed write which you can handle in the job.