Author: Elias Karthan, COO
Modern Data Architecture is a term I have been hearing more and more over the past year. You used to hear terms like Hadoop and Big Data, now it’s Modern Data Architecture. What exactly is a Modern Data Architecture, and, more importantly, do you need one? What makes a Modern Data Architecture different from existing reporting and analytic environments?
What is a Modern Data Architecture?
A quick Google search returns things like Hadoop, Streaming, NoSQL Databases, Microservices, Containers, Shared Assets, Security, Common Vocabulary, and Data Stewards. All these things are components used to create a Modern Data Architecture. I define a Modern Data Architecture as technology, processes, and principles combined to create an environment that will support an organization’s operational reporting, dashboards, data discovery, and advanced analytics needs.
Do You Need a Modern Data Architecture?
Organizations of every size in every industry are starting to realize that data matters and it’s quickly becoming one of today’s most valuable resources. The data environment is changing rapidly, and we now have data coming at us from all directions in the form of IoT, sensor data, social media activity, geospatial, logs, operational data; the list goes on and on. Organizations need to understand what the business needs to stay competitive and leverage as many of these data channels as possible to achieve operational efficiency and create competitive advantages. Ask yourself these questions:
- Do you have an integrated view of your customers (IVOC)?
- Do you know what your customers and competitors are saying about your company?
- Can you analyze your customers’ behavior to influence their future behavior?
- Can you apply business rules to detect abnormalities and take action?
- Can you support business decisions with information derived from historical data?
Companies say they are doing analytics, but the questions above can highlight whether or not they’re doing them effectively and efficiently. Often, we find that analytics is fragmented across the enterprise in things like Excel, R or Python and written by users or business specialists using data downloaded from multiple enterprise data sources. These are the efforts that need to be supported and automated by the enterprise instead of by individuals. Organizations need to understand existing capabilities and identify gaps in supporting current and future business needs and then start to modernize or enhance their existing data architecture to meet these needs.
Modernizing Your Data Architecture
The challenge is understanding how to leverage new and existing technologies to achieve operational efficiencies and competitive advantages. The process for accomplishing this is not the same for all organizations because it depends on a company’s objectives, culture, budget, and where they are in the maturity life cycle. The answer usually isn’t just a traditional data warehouse or just new technologies, it’s a hybrid of both.
The first step in modernizing your data architecture is assessing your current environment. You need to understand existing capabilities and identify gaps in supporting the business. Once you understand the gaps, create a future state architecture and a reference architecture including the technologies, processes, and principles that are the right combination for your organization to meet its reporting and analytic needs. Find use cases that can show value and use these to move your data architecture towards the future state. Every organization is different, so depending on your environment it could be small steps like cataloging and documenting existing data stores, extending an existing data warehouse, or introducing a new visualization tool. For other organizations it could be implementing a distributed data platform, creating a Data Lake, or introducing more advanced analytic tools.
The important thing is to understand the business’s needs, create a plan, and begin to take steps towards fulfilling those needs. According to Gartner, Analytics has the most potential to change the organization in the next five years. So, do you need to modernize your data architecture? It’s pretty safe to say we all do.
Related Posts
