Error records in DataWarehouse

sdfasih · Post by **sdfasih** » Wed Aug 02, 2006 10:25 am

Hi,
I would like to know what type of errors I can encounter apart from reject records from business rules when loading data from staging to datawarehouse.
Thanx.

kris007 · Post by **kris007** » Wed Aug 02, 2006 10:33 am

A lot depends on what database you are loading your data into and the nature of your source data and so on. Too big a question to answer. Search is your friend

kumar_s · Post by **kumar_s** » Wed Aug 02, 2006 9:47 pm

BTW: The records getting reject for a business rule constraint shouldnt be a technical error in all cases.

ray.wurlod · Post by **ray.wurlod** » Wed Aug 02, 2006 10:11 pm

Connectivity failures, including network, database server, authentication errors (e.g. wrong password) and more. These should be immediately fatal.

Violation of security and/or integrity constraints (most usually primary key uniqueness constraints), which should be prevented in your design.

Property or object does not exist - where "they" have changed the target table but that information has not propagated through to the ETL design team. These should be fatal, and cause recriminations between you and "them".

Scheduled or unscheduled database activity, such as backups and bulk loads, for which they have quiesced the database without informing ETL Operations group. Again, a management/communication failure which ought never to have happened. Classed as "bitter experience".

To name a few.