Measuring Innovation Boards
When large organizations develop innovation programs, they typically adopt metered funding as a way to resource these new innovation projects. This metered funding — step-by-step allocation of resources based on a series of data-driven goals and milestones — is usually governed by an innovation board.
Whether they’re called innovation boards, investor boards, venture boards, growth boards, or innovation steering committees, it all refers to that select group of people responsible for allocating funds to your company’s innovation projects. It’s a fundamentally different way of allocating resources because the decision-making process is not based on an executive’s opinion, but on data gathered by an innovation team directly from the market.
This new system of governance requires considerable resources just to implement. Teams are trained on modern entrepreneurship principles like lean startup, agile, and design thinking. Processes are designed. Executives take expensive trips to Silicon Valley to see how the startup and venture capital world operates.
But despite fostering a data-driven culture focused on objective metrics, organizations often don’t use data to measure their own performance in how they fund innovation. If innovation boards are making bad decisions and funding mediocre projects on the basis of abstract opinions not grounded in data, no amount of heroic entrepreneurialism is going to produce good results.
This is not about measuring the innovation projects themselves, which are ultimately measured by their ROI and team-based measures such as experiment velocity and insight velocity.
This is about measuring how effective innovation boards are at making decisions. If we don’t measure whether or not our innovation investment decisions are being made in the right way, then we cannot improve the process and eliminate bad bets over time.
So how do we measure if our investors are making decisions the right way?
Decoupling Decisions from Outcomes
It’s important to distinguish between measuring the outcome of the decisions and the decision-making process itself.
I could make a reasonable decision that I want to visit Australia and dive the Great Barrier Reef. Maybe I plan my vacation around that. But if I’m unable to take the trip because it’s 2020 and the world is an epic dumpster fire, that is not a failure in my decision-making process.
There is no perfect crystal ball for making decisions. If there were, we wouldn’t need an innovation process. We’d just build the right thing every time.
We are essentially placing bets, and sometimes those bets are not going to pay out. A good investor board, like good poker players, can make great decisions but still draw bad cards. A bad investor board can get lucky.
If we place enough bets, we should see that over time we win more than we lose. But an individual bet won’t tell us much about our decision-making process, let alone whether or not we correctly followed a consistent decision-making process at all.
Unfortunately, the ROI of a project is only loosely correlated with the quality of the decision-making that got it there. And the margin of error on that ROI can be very, very high. To compound that, ROI is such a lagging indicator that it’s not useful for giving timely feedback to the innovation board members. It might take years to see any ROI on a truly transformational innovation, so that data is not going to help improve our decision-making process.
Problems with Scale and Humans
The importance of measuring and improving the decision-making process becomes compounded at scale.
With one innovation board, it may seem fairly easy to take a qualitative approach and just monitor our decision-making behavior. As an external member of several innovation boards, I have been asked to give an impartial view of the board’s performance, so I do believe this approach is valuable.
But qualitative observation isn’t viable for rolling out a dozen innovation boards across multiple geographic regions and time zones. There are only so many 4 am meetings I am willing to take as an external board member.
More importantly, different innovation boards within different cultures or different departments with different goals may opt to make decisions in different ways.
One innovation board may act as more of an advisory board for the CEO, who makes all of the investment decisions themselves. Another board may be very democratic and vote on investments. Yet another may act as a group of individual angel investors. Another may be consensus-driven.
(Side note: If all of your innovation investments are consensus-driven, you’ll probably never invest in anything actually risky. In other words, you’re not actually investing in innovation.)
Each of these approaches has its merits, but we need to be able to tell which innovation boards are producing better end results, and how they are actually making decisions, so that we can transfer best practices from one innovation board to another.
A qualitative assessment of the decision-making process can hide other flaws beyond the lack of scalability. Remember that an “objective” observer is still a human observer, with their own hidden biases and irrationalities. I like to think that I’m aware of my own biases and can compensate for them, but everyone has blind spots, and you can never see your own.
Decision-Making Metrics for Innovation Boards
Having objective measures to complement subjective impressions can only improve your decision-making.
A viable innovation board needs to make their decision criteria explicit, and then have a systematic way of measuring the outcomes of the process. This encourages discipline in your decision-making, improving it over time.
There are three metrics I strongly advise innovation boards to prioritize in evaluating their own behavior:
% of funded projects that are strategically aligned.
The first must-have metric is whether or not our decisions align with the company strategy. Unfortunately, many innovation decisions are made in isolation without considering the big picture.
If an innovation team receives funding from the board and invents a new plant-based hamburger product that consumers genuinely seem to find interesting, that’s great! Success!
But if we happen to be a mining consortium that sells rocks, it’s unclear how exactly we can launch a consumer-facing hamburger business. This might be a great product for McDonalds, but not for us.
So how exactly did this project get funded in the first place?
Company strategy is often poorly defined and ambiguous. So it’s important to state the obvious; If there is no innovation strategy, none of our decisions will be strategically aligned.
To put it simply, if the strategy indicates four clear investment areas, and the innovation project doesn’t fit any of them, that’s not strategic alignment. Our strategy should indicate what domains and arenas we’re going to invest in and how much goes into each area. This strategy should include a bucket of investment for unexpected opportunities that fall between pre-defined strategic pillars.
It could be that there is a unique opportunity to provide healthy, ready-to-eat hamburger products to our mining crews and turn it into a unique B2B business model in the mining industry. None of our existing business unit VPs may be interested, but it could be an opportunity for something truly new and transformative. We shouldn’t neglect those opportunities, so our strategy should have room for them.
If our innovation program is getting set up before there is a clear innovation strategy, we can ask ourselves, “Where does the project fit in the company?” There should be an obvious choice about where to absorb this innovation and who should oversee it. Implementing a new digger would clearly fit under mining operations, while a novel, drone-based shipping system would fall to the logistics department.
If the project is truly disruptive but there is no existing business unit that makes sense to house it, then there should be an executive sponsor with enough authority to establish a new business unit if that project should succeed. Essentially, you need someone who would authorize the project to scale outside of all existing business units.
Without a potential home, projects should not be funded. This metric should always be close — if not equal to — 100%. If it isn’t, there are two possibilities:
- There is no strategy.
- No one knows what the strategy is.
- The strategy is bad, everyone knows it, and they are ignoring it.
(Tip: To make it easy to measure and ensure strategic alignment is close to 100%, just include a space on any decision template for department / executive sponsorship. If it’s blank, the decision is not strategically aligned. Innovation boards tend to pick up on this pretty quickly, and appropriately cancel projects that can’t possibly get sponsorship.)
% of funded projects that are funded with money, people, and time.
This is a must-have metric for me.
Many innovation projects are given money, but the people on the original innovation team are not assigned to the funded team. Or if they are assigned, they are working on nights and weekends.
This is just a disaster.
For starters, it is blatantly unfair to team members. We cannot just expect innovators, no matter how passionate they are about the idea, to indefinitely work for free. Why should they put effort into a project they are not actually being paid to do?
Secondly, running multiple projects at once is incredibly wasteful. If people are working on an innovation project at 2am on Wednesday night, it’s not going to be their best work. It’s better to have one hour of focused, energized work in full daylight than four hours in the dark fueled by caffeine and anxiety.
(Side note: I know, I know. Some of your best work is done fueled by caffeine and anxiety, but there’s plenty of data out there that says otherwise. Google “task switching” and save your more rigorous objections for the comments section.)
Thirdly, the time the investor boards assume is going into the project might be hijacked by the hidden investor — the direct line manager who has other priorities.
Lastly, and most importantly, money is the smallest part of the investment decision.
Modern innovation projects typically only require small amounts of cash for funding. Typical decisions on early stage projects could be anywhere from $1,000 to $50,000, but are often closer to $5,000.
But the cost of the people can be 30 times that.
If a project is funded for 3 months with 5 team members at an average salary of $10,000 each, that’s $150,000 without even considering possible opportunity costs.
This is like buying a sandwich for $10 and putting our credit card down without realizing there’s a 300% convenience surcharge. We’d be rightly pissed off at the deli.
This metric should also be close to 100%. Decisions made without counting all the money are bad decisions.
(Tip: Adding fields for this on a decision-making template can usually encourage compliance from an innovation board very quickly.)
Innovation Board Retrospectives
% of innovation board meetings that include a retrospective.
Lastly, innovation boards should run retrospectives. If investors run retrospectives on their decision-making process, it’s likely that the process will improve over time.
It’s always possible that a retrospective becomes a group pat on the back, but experience shows this is less likely when boards (and teams) consistently run them. There is always something that can be improved, and the discipline of consistently running retrospectives generally reduces the mind’s ability to ignore bad news.
When everyone is getting and giving feedback for improvement, criticisms become less personal and easier to swallow. Everyone makes a commitment to improve, and the quality of the retrospective goes up over time.
Without retrospectives that review and critique previous innovation decisions, those decisions are not likely to improve.
Again, this metric should be close to 100%, and simply implementing the metric will generally increase compliance.
Keep Measurement Lightweight
Make it easy on yourself. Don’t make the measurement so complicated that nobody is willing to be measured. If you’re requiring each investor or each investor board to write a 20-page essay on their decision-making process, they’re simply not going to do it.
The three metrics suggested here are very basic — and not the only possibilities — but they are linked to critical behaviors and easy to get started.
They are also easy to screw up.
I’ve seen plenty of boards skip retrospectives due to poor time management, and others get excited about an idea without considering how much of the overall strategic portfolio has already been allocated to the same investment area.
Eventually an experienced board may decide to add more complex metrics, such as whether or not a checklist of cognitive biases was used to eliminate blind spots, or whether or not there was a devil’s advocate promoting strenuous debate. But measuring a few simple things consistently is more effective than measuring a dozen things erratically.
What’s important to remember about your innovation board’s decision-making process is that it IS a process. All processes require the discipline to walk through the steps and measure performance along the way. If executives expect this discipline from their innovation teams, then they need to be disciplined about how they measure themselves.
- Measure decision-making to improve it.
- Lagging indicators like ROI cannot be effectively used to improve decision-making.
- Measure strategic alignment, complete decisions, and retrospectives when evaluating your innovation board decisions.