How to design job

Vengadeshwaran v.j · Post by **Vengadeshwaran v.j** » Mon Jun 30, 2008 1:40 am

Multiple files containing customer data comes from 10 countries. Each file contains exactly the same format. The same customer may exist in each file. But target should have only one record. The files may come any time. So it is possible that two files need to be processed simultaneously

1.In This Situation What are the job designs issues that you need to consider
2. How you would handle the flow of jobs

keshav0307 · Post by **keshav0307** » Mon Jun 30, 2008 2:07 am

you can design a single job and run it on multiple instances.

i had similar requirement where 120 different applications sending data file, but in same format.

on target you can use upsert method to load.
also you can find many way to to it, if you search in this forum.

ArndW · Post by **ArndW** » Mon Jun 30, 2008 2:14 am

Can you not read all the files together and merge/sort them in your job?

keshav0307 · Post by **keshav0307** » Mon Jun 30, 2008 2:17 am

oops forgot to mention that i had requirement to validate each file individually. but still we manage to design the jobs to process multiple files in a single job/sequence in single run.

ArndW · Post by **ArndW** » Mon Jun 30, 2008 2:24 am

Process the files inidividually and write the data into a DataSet using "append", then process the DataSet in a final job.