Data Integration’s Accomplices
In my post about Data Integration I came to a suitable understanding of the complete meaning of Data Integration, beyond ETL. But I felt I would be remiss if I did not mention Meta-Data and Abstraction.
Meta-data
Simply put, Meta-data is information about the data. It’s essential to have this when you claim that your Marketing campaign resulted in 200% increase in sales in a particular region, and your boss says “Where the heck did that number come from?!”. Your meta-data will show that it’s based on sales revenue numbers, and based on a 4 week moving average.
Abstraction
Put an extra layer between the user and the data. You may have heard of ‘Universes’ and ‘Cubes’ that use this concept. But isn’t this just another level of complexity? Why bother?
- So the data makes sense to the user
- So you can change a data source without breaking anything
Imagine that you are putting together a new bicycle. Would the instructions read “mount part#9889yd on #u9898c with #2929″? I hope not! Any responsible manufacturer would instead write “mount the front wheel on the front forks with the wrench provided.” The customer doesn’t have to search for parts, and the manufacturer doesn’t have to re-print instructions if they change wheel suppliers.
It’s the same with Business Intelligence. Users and analysts who write reports want information that has meaning. Its important for trust, and for audit-ability. Also, meta-data speeds up report development, since you always know what you’re working with. If you don’t have what you need, you can easily track back to the same source data and make new calculations. At the same time technology owners don’t want to have to worry about breaking all those reports if they change the underlying systems. If they do change a system, they simply have to rebuild the connection to the abstraction, and not to the 1000′s of reports that are built from it.
Meta-Data and Abstraction are key to successful Data Integration. The major vendors know this, and include tools for managing these concepts when they talk about their DI offerings. For instance SAP BusinessObjects Data Integrator uses both.
-IainR
Sr. Marketing Manager – The BI Builders
Related Posts