September 1, 2024 Author: Matthew Renze

How do you maximize the likelihood of your startup’s success?

A startup is a hypothesis — it’s a belief that your business will be successful because of x, y, or z.

To validate this hypothesis, we run a series of experiments to test a set of smaller sub-hypotheses.

For each sub-hypothesis (below), you and your stakeholders are attempting to convince yourselves that there is sufficient evidence to support each hypothesis so you can move on to the next stage of your startup.

When testing a sub-hypothesis, you should all agree on a single metric to track and an acceptance criterion for your hypothesis. For example, “if at least 10 out of 100 invited customers sign up to use our solution, then we accept that our customer acquisition hypothesis is true.”

Then, you run an experiment to gather data about your key metric to see if you have enough evidence to support your hypothesis.

If your key metric exceeds your acceptance threshold, then you provisionally accept that your hypothesis is true and move on to the next hypothesis. If not, you use feedback from your experiment to improve your solution — or you pivot to a different idea.

So, to help you validate your startup hypothesis, here are the individual sub-hypotheses that you should test — in the order that you should test them.

Problem Hypothesis

The problem hypothesis answers the question, “Do our potential customers actually have this problem?” For example, we might hypothesize that “People struggle to learn new tech skills.” Then you develop an acceptance criterion (e.g., if at least 20 out of 100 people we interview say that learning new tech skills is a significant problem they face, then we will accept our hypothesis is true.”)

To test this hypothesis, we run an experiment using either one-on-one interviews or a survey. We ask each participant a small number of questions (e.g., 3-5) to determine whether they experience the problem we’re attempting to solve. If a high enough percentage (e.g., 20%) agree that they experience the problem, our acceptance criterion is met, and we move forward to our next hypothesis.

As a startup, your goal is to solve a customer’s problem by providing a valuable solution. However, many startups start by building a solution to a hypothetical problem first—then discover later that the problem either doesn’t actually exist or isn’t big enough to generate sustainable revenue. This common but costly mistake is easily avoidable by validating your problem hypothesis as your first step.

Solution Hypothesis

The solution hypothesis answers the question, “Does our solution actually solve our customer’s problem?” For example, our hypothesis might be, “An AI tutor would help people learn new tech skills.” Our acceptance threshold might be, “If 50% of potential customers we interview agree that our solution would solve their problem, then we will accept our hypothesis is true.”

To test this hypothesis, we perform a second round of interviews or surveys. We propose our solution to potential customers (those who experience the problem we described) and ask them if they believe our solution will solve their problem. If we exceed our acceptance threshold, we move on to the next hypothesis. If not, we use their feedback to improve our solution and ask them again.

At this point in the product/solution development, we don’t want to start building our solution yet. Rather, we just need to communicate the key ideas of our solution to elicit feedback from interviewees. To do so, we create a few quick slides, diagrams, wireframes, mockups, or an explainer video. Our goal is to do the simplest thing possible to get feedback on our potential solution.

Acquisition Hypothesis

The acquisition hypothesis answers the question, “Will our customers actually use our solution?” For example, our hypothesis might be, “People will use our new AI tutor to learn new skills.” Our acceptance criteria might be, “If 5% of people we email sign up to try our service for free, then …”

To test this hypothesis, we create a Minimum Viable Product (MVP). An MVP is the simplest thing we can build that will provide the key value proposition of our solution. We offer a free or low-cost version of our MVP to a few select customers (called “friendlies”) who are willing to overlook all of the issues of a beta version of our solution and provide honest feedback to help us improve it.

Then, we determine whether enough potential customers start using our solution to solve their problem. If we meet our acceptance criteria, we move on to the next hypothesis. If not, we use feedback from customers who fail to start using our solution to determine what we need to improve to increase our customer acquisition going forward.

Retention Hypothesis

The retention hypothesis answers the question, “Will our customers continue using our solution?” For example, our hypothesis might be “Our existing customers will use our AI tutor (on average) 5 hours per week for at least three months”. We create an acceptance criterion based on these figures.

To test this hypothesis, we measure how long our customers use our solution, how frequently they visit our site, or a similar metric. We determine how many of our customers continue to use our solution on a regular basis. If a customer stops using our solution, we reach out to determine why they stopped using it and use this feedback to improve our solution and, thus, our customer retention.

Once we’ve met the acceptance threshold for customer retention, we move on to our next hypothesis. If not, we use their feedback to continue improving our solution until it meets our customers’ needs, and they continue using it regularly.

Revenue Hypothesis

The revenue hypothesis answers the question, “How much will our customers pay for our solution?” For example, our hypothesis might be, “Half of all potential customers will pay \$9.99 per month for our AI tutor.” We then construct our acceptance criterion based on those numbers.

To test this hypothesis, we start charging customers real money to use our solution. If we exceed our acceptance threshold, we might consider increasing our price. If we are below our threshold, we either need to lower our price or add more features to justify our higher price point.

It might seem strange to delay testing your revenue hypothesis until this late in the startup process. However, if you provide something that makes or saves people money, then someone will pay you up to the amount of money your solution makes or saves them. All of the previous experiments ensure that you are building something of value. The revenue hypothesis determines how much value you are providing.

Scale Hypothesis

The scale hypothesis answers the question, “Can we scale our solution to build a sustainable business?” For example, our hypothesis might be, “We can scale our AI tutor to support 1 million customers per year.” We create our acceptance criterion based on the scale we need to make a profit. This is the point where our marginal revenue (i.e. revenue per customer) exceeds our fixed and marginal operating costs.

To test this hypothesis, we rebuild the MVP using software architecture best practices. In some cases, the MVP can be evolved into a scalable architecture. However, in most cases, it’s best to completely scrap the MVP and rebuild it from scratch using a more scalable and maintainable architecture.

When you build an MVP, you optimize for learning. You want to know that your solution is right as quickly as possible, at the expense of scalability and maintainability. However, when you build a sustainable solution, you optimize for scalability and maintainability. These two goals are in opposition — you can’t do both simultaneously.

In addition, the skills necessary to build MVPs vs. sustainable solutions differ significantly. Employees are typically good at either one or the other but not both. So, your startup employees might not be the right fit for running your business at scale. You might need to move them to other projects or let them go. It’s a tough call. However, it’s often necessary to ensure you have the right people for the next phase of your business — sustainable growth.

Repeat

Once you’ve validated all of the sub-hypotheses in your startup hypothesis, you are ready to move on to the next phase of your business — sustained growth.

However, you can apply this type of hypothesis testing to each new product, service, or major feature you provide your customers. For each new product, service, or feature, you test the problem, solution, acquisition, retention, revenue, and scale hypotheses.

By continuously repeating this process, you ensure that every aspect of your business is rigorously tested and validated. This sets the foundation for ongoing success and sustainable growth.

There’s much more to creating a startup than simply validating these six hypotheses, though. If you’d like to learn more, I recommend reading The Lean Startup and Lean Analytics. These books provide valuable insights on using data science practices to increase the likelihood of your startup’s success.

In addition, if you’d like to learn more about using AI, ML, and data science to test your startup hypothesis, be sure to check out all of my online courses, videos, and articles.