Quick Answer: When measuring innovation, forget burn rate and deployment speed in isolation — what matters is iteration velocity through the complete Build-Measure-Learn loop. As product managers, we need to recognize that iteration time includes not just building, but getting the product in front of enough customers to learn something meaningful and making a decision on what to do next. A startup with 30 days of cash that iterates daily outperforms one with six months of runway that never completes a single learning cycle.

Measuring Innovation- Burn rates are meaningless in isolation. We should remind ourselves that iteration time is a critical aspect of starting a new business, and there are a few key misconceptions that we may have been guilty of:

• Our burn rate is the amount of money we’re spending each month.
• Iteration time is the amount of time it takes us to deploy a new version of our product.
• Runway = Cash on hand / Burn rate

These are just wrong. Or at the very least, not useful. Ready for takeoff? Traditionally our runway is the amount of time we have to get our company to positive cash flow, a.k.a. takeoff. Once we run out of money, the game is over. Runway = Cash on hand / Burn rate (This is a known formula of measuring innovation, but I pulled this expression of it from Eric Ries’ excellent blog post, which makes all the points in this post much more eloquently.) What is our actual burn rate? Our burn rate is the amount of money the company is spending each month, but this is an inaccurate representation for many startups. Very often, we ignore the opportunity cost. People need to eat. Personal expenses are part of our burn rate, regardless of whether or not we’re marking an IOU in the company books. Furthermore, there’s a lot of money that we could be earning instead of working on our unpaid startup. Even if we’re funemployed, we could be earning loose change working at Burger King. Those are our opportunity costs. When trying to make a decision as to how much effort we’re going to put into our company (measuring innovation), the potential rewards of our company need to be weighed against the costs. If we are miscounting those costs, we might be sinking a lot of effort into a bad decision when we could be landing a cushy job pushing buttons in a box somewhere. To be completely clear, even if our cash burn rate is $10 a month for hosting services our actual burn rate is much higher. Our time is valuable! Similarly, our cash on hand is the cash from any source that we’re using against the burn rate. So if we’re burning savings, that’s our cash on hand even if that cash is not in the company bank account. Don’t make bad decisions by ignoring money we’re investing month by month. A better way to think of Runway is: If we only have 30 days cash on hand but can iterate once per day, we’re in a far better position than a company with six months of cash that takes 6 months to build its product. In fact, the company that has six months of cash on hand and just manages to build its product in that time has zero iterations because it has neglected to budget time to get users and feedback. Time to learn Iteration cycles are not just the amount of time it takes to deploy code to the server (or for a physical product, the amount of time to create a new version of that product). The most important thing for a startup is to discover a repeatable business model- measuring innovation, not deploy code or create products. So the point of our first few iterations is not necessarily to sell the most units (although that’s certainly a good sign,) but to learn as much as possible about the business model in the least amount of time. Pushing code to the server is not learning. In order to learn anything, we have to get the product in front of enough customers to get a meaningful amount of information. In addition,measuring innovation should be considered as a decision must be made on what action to take for our next iteration. In other words, we must complete the entire Build Measure Learn loop. So, if we need statistical information about user behavior and push code to the server every day, but we only get 150 visitors a week, the iteration time is minimum eight days. Not one day. Similarly, if we push code to the server every day and get 1000 visitors a day but take two weeks to make a decision because the executive product management panel has to debate, then our iteration time is 15 days minimum. Focus on bottlenecks The most typical example of this is spending six months building a product when there is no confirmed demand. After six months of building, not a single customer has seen the product. Iteration length? Infinite. We might then go out and raise a seed round based on a prototype. (Note: Amazingly, this can sometimes be done without any evidence of customer demand, since there are no facts or metrics to prove that the product won’t work. Investors can be just as dumb as entrepreneurs; we shouldn’t take blind investment as a sign that we’re on the right track.) Our startup now has enough cash to last another 6-12 months and hire another few engineers to speed up the iteration cycles. Why? Because a faster engineering team will get to market faster with the perfect product. Real iteration length? Still infinite. A lack of metrics is the ultimate bottleneck I have personally been involved in three startups that have failed to measure progress against objective reality. I have measured the wrong thing; I have not measured at all. It is beyond embarrassing. Even when it’s a simple thing to do, sometimes metrics are an afterthought in the build process. This is a killer. Because even if we’re pushing to live customers daily and people are occasionally buying the product, we are learning nothing! How do we know they like it? Are they using the product after they purchase it? Perhaps they buy it and then think it sucks. Or do they simply not know how to install it? We will never know unless we have metrics. Quality counts When searching for metrics, it’s important to realize that qualitative data is still data and can be turned into useful information. Customer development interviews can produce very valuable feedback even when there is no product. One of our best successes with customer development was performing 51 iterations on our startupSQUARE site design using nothing but Photoshop. We don’t need to spend four months building an internal analytics system to get data. We don’t even need one line of code. If we really want to reduce our iteration time, we should set an objective to get real customer feedback twice per day. Use real information and real text, and put mockups in front of real customers. Check out these resources for additional information on gathering qualitative data through user research.

Frequently Asked Questions

What does measuring innovation actually mean for startups?

Measuring innovation means tracking how quickly we complete full learning cycles — not just how fast we build or how much cash we burn. As product managers, we need to measure our ability to get products in front of customers, gather meaningful feedback, and make informed decisions. The real metric is iteration velocity through the entire Build-Measure-Learn loop, not deployment speed or monthly spend in isolation.

Why is burn rate misleading when measured the traditional way?

Traditional burn rate only counts company expenses like hosting and salaries, but it ignores opportunity costs. As founders, our personal expenses, savings we’re depleting, and the income we’re forgoing all contribute to our true burn rate. Even if our company only spends $10 a month on hosting, our actual burn rate is much higher because our time is valuable. Ignoring these hidden costs leads to poor decisions about whether to continue pursuing a venture.

How should startups calculate iteration time correctly?

Iteration time is the total time to complete one full Build-Measure-Learn loop — not just the time to push code. If we deploy daily but only get 150 visitors per week, our minimum iteration time is eight days because we need enough data to learn from. If decision-making takes two weeks due to internal debates, that’s our real bottleneck. We need to account for build time, sufficient customer exposure, and the time to decide on next steps.

Why is a lack of metrics the biggest bottleneck for startups?

Without metrics, we’re learning nothing — even if we’re shipping to live customers daily and people are occasionally buying. We won’t know if customers actually like the product, if they use it after purchase, or if they can even install it. A lack of metrics means our effective iteration length is infinite because we never complete the learning step of the cycle. As product managers, we should treat metrics as essential infrastructure, not an afterthought.

Can you measure innovation without writing any code?

Absolutely. Qualitative data from customer development interviews counts as real, actionable information. We can run meaningful iterations using nothing but mockups — one team completed 51 design iterations using only Photoshop before writing a single line of code. If we want to reduce iteration time, we should aim to get real customer feedback on realistic mockups twice per day rather than spending months building an analytics system first.

Comments

Loading comments…

Leave a comment

Name *

Email * (not published)

Website (optional)

Comment *

Leave this blank

Post Comment