Dear All,
We are going to conduct a review of an ETL implementation soon. Would like to know if somebody has gone through a similar experience and what important areas we should be focusing first.
Our plan is to review certain areas such as:
1. Reviewing movement of data from landing to staging and then target area.
2. Restartability and scheduling of jobs (internal/external).
3. Error handling.
4. DataStage Implementation.
5. Initiation of ETL.
etc.
Thanks,
Saad
Insight for Project Review
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Welcome aboard.
This is, of course, a huge topic. Many of us here have the experience you mention, often more than once.
IBM has a specific methodology that they use, called ITERATIONS, which begins with fundamental questions like "is the enterprise ready for a data warehouse?". You might do well to contact them, I don't know whether they sell the methodology separately or only as a service. But it is a good, structured and - above all - self-documenting (mainly using Microsoft Office products) set of techniques.
Or involve an experienced consultant in the evaluation phase. The cost will easily be justified in the amount of time not wasted because you have leveraged that experience.
This is, of course, a huge topic. Many of us here have the experience you mention, often more than once.
IBM has a specific methodology that they use, called ITERATIONS, which begins with fundamental questions like "is the enterprise ready for a data warehouse?". You might do well to contact them, I don't know whether they sell the methodology separately or only as a service. But it is a good, structured and - above all - self-documenting (mainly using Microsoft Office products) set of techniques.
Or involve an experienced consultant in the evaluation phase. The cost will easily be justified in the amount of time not wasted because you have leveraged that experience.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Ray,
Thanks for sharing your views. Actually by 'Review' i mean that we are going to do the Audit of an existing ETL implementation. We would be looking into certain areas as mentioned in my email and then recommending based upon our findings.
What I would like to know are other related areas of importance and which should be analyzed carefully. Any insight would be highly appreciated.
Thanks,
Saad
Thanks for sharing your views. Actually by 'Review' i mean that we are going to do the Audit of an existing ETL implementation. We would be looking into certain areas as mentioned in my email and then recommending based upon our findings.
What I would like to know are other related areas of importance and which should be analyzed carefully. Any insight would be highly appreciated.
Thanks,
Saad
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You have missed probably THE most important questions, if it's a review of an existing implementation. These are two.
1. Are the end-users' requirements being met and, if not, why not and how could they be?
2. Are the key performance indicators of the ETL being met (usually these are expressed in terms of successfully meeting the available time windows for the extraction and loading phases)? Again, if not why not, and is there anything that can ameliorate this situation.
If there are known future plans for the enterprise, these too can be accommodated in the review as well as in ongoing ETL development.
There are many other aspects. You may be interested in reviewing what my business offers in the way of a DataStage health check.
1. Are the end-users' requirements being met and, if not, why not and how could they be?
2. Are the key performance indicators of the ETL being met (usually these are expressed in terms of successfully meeting the available time windows for the extraction and loading phases)? Again, if not why not, and is there anything that can ameliorate this situation.
If there are known future plans for the enterprise, these too can be accommodated in the review as well as in ongoing ETL development.
There are many other aspects. You may be interested in reviewing what my business offers in the way of a DataStage health check.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You have missed probably THE most important questions, if it's a review of an existing implementation. These are two.
1. Are the end-users' requirements being met and, if not, why not and how could they be?
2. Are the key performance indicators of the ETL being met (usually these are expressed in terms of successfully meeting the available time windows for the extraction and loading phases)? Again, if not why not, and is there anything that can ameliorate this situation.
If there are known future plans for the enterprise, these too can be accommodated in the review as well as in ongoing ETL development.
There are many other aspects. You may be interested in reviewing what my business offers in the way of a DataStage health check.
1. Are the end-users' requirements being met and, if not, why not and how could they be?
2. Are the key performance indicators of the ETL being met (usually these are expressed in terms of successfully meeting the available time windows for the extraction and loading phases)? Again, if not why not, and is there anything that can ameliorate this situation.
If there are known future plans for the enterprise, these too can be accommodated in the review as well as in ongoing ETL development.
There are many other aspects. You may be interested in reviewing what my business offers in the way of a DataStage health check.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.