Generate Metadata from File, then load second file?

wwalker · Post by **wwalker** » Wed Jan 28, 2009 9:19 am

Hi guys

Just got a requirement to do the following:

Receive two files, one with the target metadata and another containing the data

- Create the table based on the supplied metadata in the first file and then insert the data in the second file.

This will need to be a generic solution, as there are planned to be many such interfaces and we would like to avoid designing one job per interface. The idea would be to have a single job to do this...

Is there a way to do this dynamically in DS Server?

Thanks in advance

Wade

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Wed Jan 28, 2009 9:23 am

Before job script to create and in job loader to load.
Need more information to answer.

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Wed Jan 28, 2009 9:24 am

Before job script to create and in job loader to load.
Need more information to answer.

Mike · Post by **Mike** » Wed Jan 28, 2009 9:30 am

Not really the DataStage "way"... but I'm sure some kind of utility job could be cobbled together. You might have to treat the contents of each file as a single varchar column and parse out what you need.

In the long run, I think you will be much better off with a job per interface.

The utility approach will be brittle and a constant source of destabilization for your application.

Mike

ray.wurlod · Post by **ray.wurlod** » Wed Jan 28, 2009 7:25 pm

I agree with Mike.

It may be easier to do in parallel jobs, by converting the metadata into a schema file, but it would still be brittle - what happens when you need to refer to some dynamically-created column name in an expression?

wwalker · Post by **wwalker** » Thu Jan 29, 2009 2:35 am

I also agree with Mike, and was also my concern. However, my client has decided to do this with another technology, so therefore, this is resolved on the DS side

Thanks

Wade