How to dynamically change metadata without changing job

[nash]

Hi,
We have a requirement where we need to identify those jobs that are impacted because of metadata change.
I know this can be achieved through Usage Analysis.

But we also want that if metadata changes for that job, the job should dynamically include the changed metadata in place of the existing meta data definition and execute without any code change to the job.

Is it possible with Runtime column propagation? Please suggest.

Can somebody please suggest some ideas on this or if you have encountered any situation like this one?

Thanks.

[Mike]

RCP could be used to some degree... but why?

You want to intentionally add a destabilizing feature to your application design?

Mike

[eostic]

Also, can you outline what the "dynamic metadata" is? Is it columns added or removed, changes in datatype, changes in arrangement/order of columns?...etc.

Some changes are handled automatically, some could be managed by things like RCP, some might require additional stages or transformations......... lots of possibilities here.

Ernie

[nash]

RCP could be used to some degree... but why?

You want to intentionally add a destabilizing feature to your application design?

Mike

The input file will have few mandatory columns and few optional columns. These optional columns may differ in each input file. The order of columns can also change

The idea is to just map the columns from source and create an output in various formats lik csv, dat, etc

We donot want to edit the job once built. There will not be any complex transformations other than few simple business rules.

Please advise if we can achieve it thru RCP
Thanks

[chulett]

Somewhat. Any columns handled by RCP will be strictly "pass-through", i.e. with zero in the way of transformations.

[nash]

Somewhat. Any columns handled by RCP will be strictly "pass-through", i.e. with zero in the way of transformations. ...

Thanks

[dsuser_cai]

Somewhat. Any columns handled by RCP will be strictly "pass-through", i.e. with zero in the way of transformations. ...

Thanks

Hi

i had worked in the same kind of situation. but mine was little different. we had a ods where all the transformations will be done and with this as a source we had another data warehouse where we need to populate the tables. we had nearly 200 target tables, but we had only 5 jobs that will handle the entire situation. We had the following conditions:

Meta data : run time propagation
SQL Query, other dates, key columns : everything will be passed as a parameter from a parameter file. if this is what you are looking for let me know, i think i can help you.

[dsuser_cai]

Somewhat. Any columns handled by RCP will be strictly "pass-through", i.e. with zero in the way of transformations. ...

Thanks

Hi

i had worked in the same kind of situation. but mine was little different. we had a ods where all the transformations will be done and with this as a source we had another data warehouse where we need to populate the tables. we had nearly 200 target tables, but we had only 5 jobs that will handle the entire situation. We had the following conditions:

Meta data : run time propagation
SQL Query, other dates, key columns : everything will be passed as a parameter from a parameter file. if this is what you are looking for let me know, i think i can help you.

[ray.wurlod]

It's not too great a step from "dynamic" to "dynamite". Beware trying to do anything "dynamic" in what is designed to be a metadata-driven tool.

[nash]

Thanks for your inputs. We are currently trying a POC for the same. Will post a query if we get stuck somewhere.