Multiple file mutiple tables

dsdeveloper123 · Post by **dsdeveloper123** » Mon Jun 25, 2007 12:44 am

Hi,

I have mutiple comma seperated files.
They vary in thier structure.
I have to read them all and place them into the necessary tables.
My problem is I have ended up writing 10 server job for the 10 input files I have. I have connected all these 10 jobs using job sequence.
But I guess this a wrong way of programming.
Is there any work around.

Thankx in advance

ArndW · Post by **ArndW** » Mon Jun 25, 2007 12:49 am

If the files and the tables have different structures then you will need 10 jobs to do this, or perhaps you could combine what you now have in 10 jobs into 1 job and skip using the job sequence {but that doesn't change much}.

dsdeveloper123 · Post by **dsdeveloper123** » Mon Jun 25, 2007 1:32 am

Thank you...

I am new to data stage so was wondering if what I am doing is right or wrong...

dsdeveloper123 · Post by **dsdeveloper123** » Tue Jul 10, 2007 7:34 am

Hi,

Can I achieve the job I mentioned in this post using parallel jobs.

chulett · Post by **chulett** » Tue Jul 10, 2007 7:40 am

dsdeveloper123 wrote:I am new to data stage so was wondering if what I am doing is right or wrong...

It's perfectly fine... aka 'right'.

dsdeveloper123 · Post by **dsdeveloper123** » Tue Jul 10, 2007 7:54 am

Ok..

So u mean using parallel jobs also I cant archive transferring 10 files of different structure using a single job.

Thank You

chulett · Post by **chulett** » Tue Jul 10, 2007 8:09 am

[cheech]U's not here man...[/chong]

No, I simply answered the original quoted question. Are you doing something wrong? No. Ten jobs to process ten different files is perfectly fine.

Just how 'different' are these different file layouts? If the differences are fairly minor, then perhaps you can cut back on the number of jobs. Or you can create one job that reads all records as a single large string and then, based on what it finds in the first X characters, decides what format it is seeing and you split the string up accordingly inside the job. Personally, I don't see the point in doing anything like that. Use ten jobs.

Server, parallel, doesn't matter - there really isn't anything magical about either product that will let you process multiple completely different file layouts in a 'single job' without a great deal of 'manual' effort.

dsdeveloper123 · Post by **dsdeveloper123** » Tue Jul 10, 2007 8:25 am

Thank u ...

You helped me clear my doubt. The files vary a lot from each other. So I guess I'll go with ten jobs.
I just wanted to be sure that I was not missing on something.