Exact use of staging tables?
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 59
- Joined: Wed Dec 31, 2008 5:49 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
They're also good to make sure that all your data has been loaded into the target data warehouse area (i.e. the "staging" area).
In the case where for example, not all your source data files where available then you could wait until all the staging jobs completed, at this point you'd know all your data is ready to be loaded into the data warehouse.
Then you can load your work tables using some transformations, the whole process happening in it's own separate environment.
In the case where for example, not all your source data files where available then you could wait until all the staging jobs completed, at this point you'd know all your data is ready to be loaded into the data warehouse.
Then you can load your work tables using some transformations, the whole process happening in it's own separate environment.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I prefer Data Sets for that (with parallel jobs). They are more efficiently loaded and unloaded than database tables.
Things like referential integrity should have been checked earlier in the ETL process - you don't need a "staging database" to do that.
![Idea :idea:](./images/smilies/icon_idea.gif)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.