After years of failure to be able to use operational data for management purposes, the concept of a "data warehouse" has been gathering great interest. At last we have the promise of making all (or at least a large part of) a company's data available in a consistent, coherent form for management reporting and analysis. A data warehouse has three main components:
- A "central data warehouse" or "operational data store", which is a data base organized according to the corporate data model.
- One or more "data marts" — extracts from the central data warehouse that are organized according to the particular retrieval requirements of individual users.
- The "legacy systems" where an enterprise's data are currently kept.
The work required to address each of these areas is quite different.
The Central Data Warehouse
The Central Data Warehouse is just that — a warehouse. All the enterprise's data are stored in there, "normalized", in order to minimize redundancy and so that each may be found easily. This is accomplished by organizing it according to the enterprise's corporate data model. Think of it as a giant grocery store warehouse where the potato chips are kept in one section, the beer is in another, and the video tapes are in a third.
As can be seen from the industy experience part of this web page, Essential Strategies, Inc. has had extensive experience producing corporate data models in a wide variety of industries. Most recently, ESI developed a corporate model that covered the requirements of several different divisions of a glass manufacturer, and then used that model to create a central data warehouse.
Each data mart is a collection of tables organized according to the particular requirements of a user or group of users. Retrieving a collection of different kinds of data from a "normalized" warehouse can be complex and time-consuming. Hence the need to rearrange the data so they can be retrieved more easily. The notion of a "mart" suggests that it is organized for the ultimate consumers — with the potato chips, beer, and video tapes all next to each other.
This organization does not have to follow any particular inherent rules or structures. Indeed, it may not even make sense. And however the marts are organized initially, the requirements are almost certain to change once the user has seen the implications of the request.
This means that the creation of data marts requires:
- Understanding of the business involved.
- Responsiveness to the user's stated objectives.
- Sufficient facility with data base modeling and design to produce new tables quickly.
- Tools to convert models into data marts quickly.
Essential Strategies, Inc. has the experience with on-line analytical systems required to do this. ESI consultants are among the best in the industry when it comes to producing models quickly, and they make effective use of CASE tools to turn those models in to data bases quickly. See Using Data Model Patterns for Rapid Applications Development for more on how ESI uses its library of patterns to expedite the process. This web page is a demonstration of ESI's ability to organize information for presentation to a wide variety of consumers.
To learn more about the relationship between the design of a data warehouse and the design of data marts, see From a Relational to a Multi-dimensional Data Base.
The task of documenting legacy systems and mapping their columns to those of the central data warehouse is the worst part of any data warehouse project. Moreover, the particular characteristics of the project vary, depending on the nature, technology, and sites of those systems.
Essential Strategies, Inc. has extended Oracle's Designer/2000 product to allow for reverse engineering of legacy systems — specifically providing the facility for mapping each column of a legacy system to an attribute of the corporate data model. Combining this with the mapping of each column in a data mart to an attribute of the data model provides direct mapping of original data to the view of them seen by users.