Traditional Data Warehouses Can’t Do it All
Photo credit: Canva
As part of my new professor gig, I teach CIS405 (Business Intelligence) at the W. P. Carey School. It’s an online course that demonstrates how students can turn raw data into intelligence, insights, analytics, and, ultimately, better business decisions.
When I think about contemporary business intelligence, data mining, data visualization, data warehousing, and analytics, I find it difficult to explain simply where one ends and the others begin. On the contrary, there seems to be a great deal of overlap among these terms and concepts. To be sure, things are muddier than ever here.
It’s more clear that ever, though, traditional data warehouses will not store every type of information relevant to an enterprise. To paraphrase Kevin Hart, let me explain why.
Limitations of Traditional Data Warehouses
I’m no expert, but I’ve worked around data warehouses (DWs) for a long time. Sure, they can store a great deal of data from different sources. Make no mistake, though: This benefit is concurrently a limitation. That is, DWs require very specific and static structures and categories. The word flexible certainly doesn’t come to mind. (This problem is more acute when organizations lack a proper data-warehousing strategy.) Collectively, these rules and decisions restrict the types of analysis that organizations can perform.
For years, data warehouses represented the best that we could do for data storage and consolidation. Not anymore. Hadoop and other NoSQL databases inarguably represent the future of data storage. Some have even that DW appliances will go the way of the Dodo. Ditto for the “traditional” single-database implementation of a DW.
Fortunately, there’s good news for those looking for more flexible data-storage, reporting, and infrastructure solutions. New tools can handle significantly faster, larger, and more diverse datasets—and offer real-time analytics in the process. (This is essential, as data streams will doubtless increase in the future with no end in sight.)
Simon Says: Get ready to embrace new data-management tools.
Brass tacks: Despite their considerable warts, I don’t doubt for a minute that mature BI and DW tools will remain in place for decades, particularly at mature organizations that have spent millions configuring them just right. Old habits die hard.
Many CXOs of mature organizations will eventually realize, however, that new data varieties, velocities, and volumes [PDF], simply won’t play nicely—if at all—with traditional data-management and -storage tools.