Page 1 of 1
Multiple file mutiple tables
Posted: Mon Jun 25, 2007 12:44 am
by dsdeveloper123
Hi,
I have mutiple comma seperated files.
They vary in thier structure.
I have to read them all and place them into the necessary tables.
My problem is I have ended up writing 10 server job for the 10 input files I have. I have connected all these 10 jobs using job sequence.
But I guess this a wrong way of programming.
Is there any work around.
Thankx in advance
Posted: Mon Jun 25, 2007 12:49 am
by ArndW
If the files and the tables have different structures then you will need 10 jobs to do this, or perhaps you could combine what you now have in 10 jobs into 1 job and skip using the job sequence {but that doesn't change much}.
Posted: Mon Jun 25, 2007 1:32 am
by dsdeveloper123
Thank you...
I am new to data stage so was wondering if what I am doing is right or wrong...
Posted: Tue Jul 10, 2007 7:34 am
by dsdeveloper123
Hi,
Can I achieve the job I mentioned in this post using parallel jobs.
Posted: Tue Jul 10, 2007 7:40 am
by chulett
dsdeveloper123 wrote:I am new to data stage so was wondering if what I am doing is right or wrong...
It's perfectly fine... aka 'right'.
Posted: Tue Jul 10, 2007 7:54 am
by dsdeveloper123
Ok..
So u mean using parallel jobs also I cant archive transferring 10 files of different structure using a single job.
Thank You
Posted: Tue Jul 10, 2007 8:09 am
by chulett
[cheech]U's not here man...[/chong]
No, I simply answered the original quoted question. Are you doing something wrong? No. Ten jobs to process ten different files is perfectly fine.
Just how 'different' are these different file layouts? If the differences are fairly minor, then perhaps you can cut back on the number of jobs. Or you can create one job that reads all records as a single large string and then, based on what it finds in the first X characters, decides what format it is seeing and you split the string up accordingly inside the job. Personally, I don't see the point in doing anything like that. Use ten jobs.
Server, parallel, doesn't matter - there really isn't anything magical about either product that will let you process multiple completely different file layouts in a 'single job' without a great deal of 'manual' effort.
Posted: Tue Jul 10, 2007 8:25 am
by dsdeveloper123
Thank u ...
You helped me clear my doubt. The files vary a lot from each other. So I guess I'll go with ten jobs.
I just wanted to be sure that I was not missing on something.
