ETL: More than Middleware?
Posted: Fri Jun 18, 2010 10:39 am
ETL: More than Middleware?
Page 1 of 1
Re: ETL is just another piece of Middleware?
Posted: Fri Jun 18, 2010 10:50 am
Re: ETL is just another piece of Middleware?
Posted: Fri Jun 18, 2010 11:00 am
How important is a core competency?
I think you have answered by question below.
Thanks and you folks are awesome.
I think you have answered by question below.
Thanks and you folks are awesome.
Posted: Fri Jun 18, 2010 11:28 am
You seem to be having this conversation with yourself... 
Posted: Fri Jun 18, 2010 11:32 am
chulett wrote:You seem to be having this conversation with yourself...
Yeah, I do like to argue with myself sometimes. But seriously, I am looking for opinions other than my own. Sorry about that.
Posted: Fri Jun 18, 2010 12:25 pm
I agree. You do like to argue with yourself. Seriously, this never works. When admins monitor DataStage without knowledge of the product then it never runs smooth. They call about things that are not a problem and ignore serious problems.
Posted: Fri Jun 18, 2010 1:38 pm
Ok. I'll bite.
It is vastly more than just "another piece of middleware." It's often the core of the entire data integration infrastructure for thousands of companies. And as that "core," represents the lion's share of intellectual capital about "the" data and metrics that drive the enterprise. Often the folks who build decision support systems (still a majority of the apps that use ETL) know more about the data, it's meaning, its relationships, and its ideosyncracies than anyone else in the organization. Their understanding is reflected in ETL (whatever tool, or 3gl code they might be using) and in every other part of the lifecycle that surrounds it......profiling, data quality, modeling, cubing services, reporting, portals, legacy data stores, and more..........lots more.....
It can't be ignored by anyone, let alone be a delivery vehicle who results are the responsibility of end users.
Sure there are configurations where ETL is just a "movement process," but even there, treating it as just "plumbing" is a continuation of the habits that got us here in the first place (with hundreds of definitions of customer, no awareness of metadata, yada, yada.).
Ernie
It is vastly more than just "another piece of middleware." It's often the core of the entire data integration infrastructure for thousands of companies. And as that "core," represents the lion's share of intellectual capital about "the" data and metrics that drive the enterprise. Often the folks who build decision support systems (still a majority of the apps that use ETL) know more about the data, it's meaning, its relationships, and its ideosyncracies than anyone else in the organization. Their understanding is reflected in ETL (whatever tool, or 3gl code they might be using) and in every other part of the lifecycle that surrounds it......profiling, data quality, modeling, cubing services, reporting, portals, legacy data stores, and more..........lots more.....
It can't be ignored by anyone, let alone be a delivery vehicle who results are the responsibility of end users.
Sure there are configurations where ETL is just a "movement process," but even there, treating it as just "plumbing" is a continuation of the habits that got us here in the first place (with hundreds of definitions of customer, no awareness of metadata, yada, yada.).
Ernie