Matthew Renze

The Data Science Hierarchy of Needs

The process for becoming a data-driven organization follows a hierarchy of needs

Author: Matthew Renze
Posted: 2019-01-15

The Data Science Hierarchy of Needs

Is your organization stuck on its journey to data-driven AI?

Many organizations find that they become stalled in their data-science journey at some point in time. Sometimes its because they haven't built the necessary foundation to transition to the next level. Other times, it's because they don't know where to go next, or how to get there.

So how did companies like Google, Amazon, and Facebook transform themselves into data-driven enterprises? Well, it definitely didn't happen overnight. Data-driven organizations are grown and evolve over time. They go through various stages of growth as they reach maturity.

These stages of growth are based upon a hierarchy of data-driven needs. Essentially, we can't get to the next stage of organizational transformation until we have sufficiently satisfied our lower (more primal) needs.

Collect

The most basic need of a data-driven organization is the need to collect data. This starts with basic data collection activities like recording transactions, logging errors, and digitizing analog data.

Then, as the company evolves, this can lead to more advanced forms of data collection. The organization may begin gathering telemetry data from applications, running experiments to create new data, and acquiring data from external sources.

Organize

Next, we have a need to organize our data. We need to get our data in a form suitable for analysis. This starts with basic data-organization tasks like transforming, cleaning, and storing data.

Then, as the company matures, this may often lead to building more robust solutions. The organization may build a data ETL pipeline, a data warehouse, or a data lake.

Analyze

Third, we have a need to analyze our data. We need to use our data to explain what's happening in our organization and why it's happening. This generally starts with basic data analysis tools, like reports, dashboards, and KPIs.

Then, as the company matures, this may lead to more powerful forms of data analysis. They may begin to incorporate data mining, descriptive analytics, and diagnostic analytics into their data-science pipeline.

Predict

Fourth, we have a need to make predictions. We want to know what will likely happen in the future and how we should respond to these potential scenarios should they occur. This generally involves more advanced types of data analysis. For example, the organization may begin incorporating predictive analytics, prescriptive analytics, and machine learning in their data-science pipeline.

Automate

Finally, we have a need to automate our data-science processes. This is where we close the data-science loop and remove the human from the process. This involves advanced technologies like artificial intelligence, deep learning, and reinforcement learning.

Automation of data-science processes, in the form of data-driven AI, is the goal of data-driven organizations. When applied properly, data-driven AI can minimize our costs and maximize our revenue. This type of AI is what sets the industry leaders apart from everyone else.

Data science is essentially a stepping stone on the road to data-driven AI. However, in order to become an AI-driven organization, we first need to become a data-driven organization.

We need to cultivate organization knowledge and adoption of these core data-science practices before we can achieve the transformative effects of modern artificial intelligence.

To learn more about becoming a data-driven organization, please check out my online courses on data science.

Thanks to the various incarnations of data science hierarchy of needs that inspired this post, including Jay Kreps, Yanir Seroussi, Monica Rogati, and of course, Abraham Maslow.

Share this article:  Share on Facebook Share on Twitter Share on LinkedIn