Handling different metadata in runtime for the same table in

dilip.datastage · Post by **dilip.datastage** » Tue Dec 11, 2007 11:30 pm

Hi,
I have a situation to handle different metadata for the same target table in incremental load. How can i procced with. should i have to change manually for different load or there is any way to handle this automatically.

ray.wurlod · Post by **ray.wurlod** » Wed Dec 12, 2007 1:34 am

Please explain how the structure of a table can change between runs. If it is changed manually (for example via ALTER TABLE statements) then there is no way to automatically synchronize your DataStage job. Recall that DataStage job designs are driven by the metadata - not vice versa.

dilip.datastage · Post by **dilip.datastage** » Wed Dec 12, 2007 5:00 am

From source i am geeting stream data. Data from source will come with different metadata. I have to handle it automatically by creating tables at run time.

chulett · Post by **chulett** » Wed Dec 12, 2007 6:48 am

Still doesn't really explain what you need to do or what exactly you mean by 'with different metadata'. Can you try again please? Perhaps with an example or three?

ray.wurlod · Post by **ray.wurlod** » Wed Dec 12, 2007 7:13 pm

In particular, how do you learn the format of the source stream, from which you hope to be able to create the target table? And what database is this? There may be a bulk loader solution possible. At the same time that you create the table you create the bulk loader control file based on the same metadata. The question remains, however, where you obtain that metadata.

dilip.datastage · Post by **dilip.datastage** » Thu Dec 13, 2007 4:50 am

Hi,
From source itself i will be getting different metadata for each run. I have to handle this in ETL not in database tables. Is there any way to handle automatically different metadata in each run atomatically through ETL job.

i heard it is possible by using XML stage. Can you suggest me on this.

Regard's
Dilip

chulett · Post by **chulett** » Thu Dec 13, 2007 7:26 am

[sigh]

No-one can help until you stop repeating the little bit you've been repeating and explain in detail the problem you are facing. What exactly do you mean by 'different metadata'? Can you provide examples of these differences?

dilip.datastage · Post by **dilip.datastage** » Mon Dec 17, 2007 11:57 pm

Hi,
Sorry for not giving clear picture of the problem.

I am using input as XML source file. Current run it has 4 columns :
ID, Name, Age, Address. I will process this .

In the next run i will get 5 columns(Metadata change) :
ID, Name, Age, Address1, Address2.

How can i handle this situation. Since i am getting files in XML can i handle it in ETL automatically. Can same job handle different metadata each time.
I heard this is possible in XML. Can my job read different metadata from XSD file and process the same.

Thanx in advance,
Dilip

dilip.datastage · Post by **dilip.datastage** » Mon Dec 17, 2007 11:58 pm

Hi,
Sorry for not giving clear picture of the problem.

I am using input as XML source file. Current run it has 4 columns :
ID, Name, Age, Address. I will process this .

In the next run i will get 5 columns(Metadata change) :
ID, Name, Age, Address1, Address2.

How can i handle this situation. Since i am getting files in XML can i handle it in ETL automatically. Can same job handle different metadata each time.
I heard this is possible in XML. Can my job read different metadata from XSD file and process the same.

Thanx in advance,
Dilip