Handling different metadata in runtime for the same table in
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 22
- Joined: Wed Aug 15, 2007 10:59 pm
- Location: Bangalore
Handling different metadata in runtime for the same table in
Hi,
I have a situation to handle different metadata for the same target table in incremental load. How can i procced with. should i have to change manually for different load or there is any way to handle this automatically.
I have a situation to handle different metadata for the same target table in incremental load. How can i procced with. should i have to change manually for different load or there is any way to handle this automatically.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Please explain how the structure of a table can change between runs. If it is changed manually (for example via ALTER TABLE statements) then there is no way to automatically synchronize your DataStage job. Recall that DataStage job designs are driven by the metadata - not vice versa.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 22
- Joined: Wed Aug 15, 2007 10:59 pm
- Location: Bangalore
Hi
From source i am geeting stream data. Data from source will come with different metadata. I have to handle it automatically by creating tables at run time.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
In particular, how do you learn the format of the source stream, from which you hope to be able to create the target table? And what database is this? There may be a bulk loader solution possible. At the same time that you create the table you create the bulk loader control file based on the same metadata. The question remains, however, where you obtain that metadata.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 22
- Joined: Wed Aug 15, 2007 10:59 pm
- Location: Bangalore
Hi,
From source itself i will be getting different metadata for each run. I have to handle this in ETL not in database tables. Is there any way to handle automatically different metadata in each run atomatically through ETL job.
i heard it is possible by using XML stage. Can you suggest me on this.
Regard's
Dilip
From source itself i will be getting different metadata for each run. I have to handle this in ETL not in database tables. Is there any way to handle automatically different metadata in each run atomatically through ETL job.
i heard it is possible by using XML stage. Can you suggest me on this.
Regard's
Dilip
-
- Participant
- Posts: 22
- Joined: Wed Aug 15, 2007 10:59 pm
- Location: Bangalore
Hi,
Sorry for not giving clear picture of the problem.
I am using input as XML source file. Current run it has 4 columns :
ID, Name, Age, Address. I will process this .
In the next run i will get 5 columns(Metadata change) :
ID, Name, Age, Address1, Address2.
How can i handle this situation. Since i am getting files in XML can i handle it in ETL automatically. Can same job handle different metadata each time.
I heard this is possible in XML. Can my job read different metadata from XSD file and process the same.
Thanx in advance,
Dilip
Sorry for not giving clear picture of the problem.
I am using input as XML source file. Current run it has 4 columns :
ID, Name, Age, Address. I will process this .
In the next run i will get 5 columns(Metadata change) :
ID, Name, Age, Address1, Address2.
How can i handle this situation. Since i am getting files in XML can i handle it in ETL automatically. Can same job handle different metadata each time.
I heard this is possible in XML. Can my job read different metadata from XSD file and process the same.
Thanx in advance,
Dilip
-
- Participant
- Posts: 22
- Joined: Wed Aug 15, 2007 10:59 pm
- Location: Bangalore
Hi,
Sorry for not giving clear picture of the problem.
I am using input as XML source file. Current run it has 4 columns :
ID, Name, Age, Address. I will process this .
In the next run i will get 5 columns(Metadata change) :
ID, Name, Age, Address1, Address2.
How can i handle this situation. Since i am getting files in XML can i handle it in ETL automatically. Can same job handle different metadata each time.
I heard this is possible in XML. Can my job read different metadata from XSD file and process the same.
Thanx in advance,
Dilip
Sorry for not giving clear picture of the problem.
I am using input as XML source file. Current run it has 4 columns :
ID, Name, Age, Address. I will process this .
In the next run i will get 5 columns(Metadata change) :
ID, Name, Age, Address1, Address2.
How can i handle this situation. Since i am getting files in XML can i handle it in ETL automatically. Can same job handle different metadata each time.
I heard this is possible in XML. Can my job read different metadata from XSD file and process the same.
Thanx in advance,
Dilip