Datawarehouse's Data Merging Considerations

holymac · Post by **holymac** » Mon Sep 25, 2006 8:02 pm

Dear All,

Would like to put forward this question as well:

Our company has a datawarehouse and we have merged with a new company that does not have a datawarehouse and now we are to combine/merge the new company's data into our existing datawarehouse.

I would like to know from everyone in this forum on what are the areas and issues/aspects of consideration in preparing for this merging activity.

Thanks in advance.

kduke · Post by **kduke** » Mon Sep 25, 2006 8:56 pm

Are these job interview questions?

holymac · Post by **holymac** » Mon Sep 25, 2006 9:02 pm

No it is not Duke. This is an actual scenario i am facing right now.

DSguru2B · Post by **DSguru2B** » Mon Sep 25, 2006 9:34 pm

For that you need to get your data modeler/data architect involved. Lay out a rough sketch of how the data is combinable. Handle cons, create extra tables for different/new dimensions, create fact/factless fact tables, or add new attributes to the existing tables.

ray.wurlod · Post by **ray.wurlod** » Mon Sep 25, 2006 11:15 pm

Look also for opportunities to use conformed dimensions. For example, typically the same time granularity will be used regardless of data source, so that the same time dimension table can be used with all fact tables.

kduke · Post by **kduke** » Tue Sep 26, 2006 7:31 am

I would look at the 2 designs and see what is in common. If you truly need to merge dimensions then you may need to use QualityStage with some address matching. You need the cleanse and standardize first.

I would pick a subject area where both are strong.

vmcburney · Post by **vmcburney** » Tue Sep 26, 2006 7:03 pm

You should also consider a master data management process. For example, where is your definitive list of customers, addresses and products? Are you decommissioning core systems? You could try and make your data warehouse the consolidated view of all information from each set of systems but you face a difficult matching, consolidation and deduplication task as you populate the DW.

ray.wurlod · Post by **ray.wurlod** » Tue Sep 26, 2006 7:41 pm

... a task much less difficult if you have QualityStage.