Experiment Engines: The Key To Unlocking Agility and Growth In Organisations Of Scale

We started ?What If! over 25 years ago. The goal was to help supertanker companies figure out how to act with speedboat levels of agility. Every year we are lucky enough to work on hundreds of engagements from our offices in the US, UK and China. Our clients are typically large incumbents operating in complex regulated and increasingly super-fast environments.

Within these supertankers an unhealthy narrative is taking shape. It goes like this: Scale has become a fatal brake on growth. Innovation is so complex that bureaucratic legacy players like us just can’t win against the new technology based platform businesses. In short: “We’re doomed!” We meet many hard-working leaders who believe this, or are starting to believe it. They are fatigued at fighting their supertanker system and really struggling to kick start growth.

So what are the options for the supertanker that dreams of being a speedboat? First, many have placed big bets on buying into the youthful promise of start-ups.  But whilst some organisations are rejuvenated, this near fetishization of entrepreneurs brings its own unique problems to others as the speedboat struggles to integrate into the supertanker. Second, many supertankers look to organised programmes to transform culture. But the obsession with change programmes is drying up as our clients find them just too slow to keep pace with the rate of market and technology change.  Finally, supertankers often have no option but to subcontract growth ideas and implementation to external vendors.  But no one really wants to hire a consultant, and their ability to deliver growth is mixed.

Large organisations need a new and more hopeful recipe for growth. And one that leverages their scale rather than faces away from it.  We need to find a way to make Big beautiful again.

We have helped our clients innovate and grow across just about every geography and sector and over thousands of engagements. When we turned 25 and we asked our experienced and passionate team (we call them ‘Inventors’) to keep their ears closer to the ground than usual.  We wanted to know what was really working for supertankers craving disruptive growth.  What were we, or anyone else doing that was making a difference?

What did we find? Big companies can be as nimble as small companies if they embrace Experimentation and they embrace it at scale.  And while small companies have a better reputation around creativity, bigger companies are actually much better placed than many of their leaders realise to reap the benefits from a test and learn culture.

Big can be beautiful because of:

• Deep pockets: Experimentation is often hard to plan and may need sustaining for longer than anticipated, certainly longer than a small business can tolerate. Sometimes a first round of Experimentation will need another and another as the opportunity is continually iterated all the way to launch and often beyond. Having secure access to funding can help Experimentation teams stay afloat
• Access: Most large organisations have deep, global access to customers, consumers, facilities and subject matter experts – the type of Experiment infrastructure that start-ups dream about. One of the main reasons an Experiment goes wrong is that it took too long to find a suitable audience to experiment with. Big companies often have ready made audiences in panels, beta test communities or colleagues.
• Scale: By sharing knowledge around Experiments across groups, regions and categories, large organisations can create a learning multiplier effect
• Human Energy: Experimentation unleashes a deep well of energy pent up in large organisations, whereas most people in small organisations are energetically maxed out

Our goal in these articles is not to comment on how native Experimenters (think Amazon and Apple) do their thing.  Instead we deliberately focus on the greater challenge which is to retrofit a culture of Experimentation into big legacy beasts. This is a place where ‘trying things out’ and on an industrial scale can feel highly uncomfortable but the rewards are worth it.

But when a big organisation does take Experimentation to its heart and repeats the Experiment process over time, across siloes, and at scale it becomes… an Experiment Engine.

An Experiment Engine has the power to transform how strategy is created, how disruptive new customer offers are hatched in market and how cultural transformation becomes a reality.

Our vision is that if enough people in large organisations can develop the competence and confidence to run and take part in Experiments then the agility of a start-up can truly be married to the scale of a supertanker. Goliath can be as nimble as David.

So you’re thinking – big claim eh?  Please hang in here while we define what we mean by an Experiment (yes we’re capitalising it) and an Experiment Engine.

What Is Experimentation?

Experimentation is the creative discipline of learning by trying something new. Experiments are really about asking ‘what if?’ but in a structured way, so that teams and leaders can feel confident moving forward in an uncertain environment.

Key elements of the Experimentation process are well known:

• Start with an hypothesis or point of view about what’s wrong with things (problem hypothesis), or how something could be fixed (solution hypothesis)
• Select a small team who are engaged with the hypothesis (they give a damn), who understand how Experiments work and are able to work autonomously
• Permission is clearly given to Experiment, with an open understanding that some experiments will ‘fail’
• Teams plan ‘just rigorous enough’ experiments. They get clear on what they’re trying to learn, the purpose of the experiment, how it’s going to work, and what outcomes they’re going to look for
• Teams work in bursts to expose customers to ideas and prototypes, try out new behaviours or operational processes. After each burst (or ‘sprint’ or ‘cycle’) feedback and data is analysed
• The team is now armed with enough information to make confident decisions about what to do next: Change tack, keep moving, scale up, or shelve their work

The Power Of Experimentation

From our experience on the frontlines on innovation we’ve heard Experimenters report the following exciting benefits:

Audacity

• And because it is anticipated that not all experiments will work people tend to explore more radical solutions to problems. There is a level of liberation that comes from knowing that the market, not internal stakeholders, holds the answer. Likewise a leader can only lead for Experimentation if they are credible around uncertainty. They need to be prepared to walk into the Boardroom and say they just don’t know the answer – safe in the knowledge they know how to Experiment, and quickly get the answer.

Because Experimentation values ‘doing’ over thinking there is no time for doubt to creep in.

Engagement

• Experimentation engages people at all levels. It’s all about getting on and doing things and good people want to stay in companies that do things, rather than plan to do things. Most C-suites appreciate the dynamism of Experimentation and generally like an opportunity themselves to get out of the office and roll their sleeves up with customers. This activity is a great alignment maker. Likewise, colleagues in middle management who are fatigued with change initiatives appreciate Experimentation as their way of making a difference. And for leaders everywhere, Experimentation enables them to practice a more distributed and empowering style of working.

Confidence

• Experiments, as we define and practice them, are designed to be easy to use and deliver tangible outcomes surprisingly fast. If Agile is an industrial strength transformation delivery approach then ‘Experimentation’ gets the popular vote as an easy to use, anytime-anywhere-anyone tool. It’s  designed to rapidly shape new thinking and take on the start-ups. Experiments are so simple and rewarding they hold a magnetic appeal for colleagues everywhere, and as they are repeated personal confidence rises. Over time this creates a critical mass of passionate Experimenters or an ‘Experiment Engine’.

Low Investment

• Some businesses will spend half a year building out a proposition before really understanding how many customers would use it or buy it whereas the beauty of Experimentation is that it can be done, and should be done, cheaply and quickly. Experimenters know that the shorter the gap between having an idea or hypotheses and getting feedback on it the better. So Experiments naturally tend to be short and sharp. I’m sure that anyone reading this now can think of several investments that should have been stopped if only the right type of thinking had been done up front; ‘if only we’d have made it real and tried it out earlier’.

We believe this magical cocktail of audacity, engagement, confidence and value for money means that every supertanker CEO should have a plan to recreate their business as an Experiment Engine at the top of their to-do list.

How Experimentation Works: In the Corridor, and the Boardroom.

Informal and unplanned

The need to Experiment can reveal itself in informal situations. Two supply chain colleagues bump into each other in the corridor. One expresses excitement about the potential to add revenue by electronically tagging all packages and sending the customer an hourly arrival countdown.  The colleagues do a quick calculation which says the financial upside could be very big but they can also see several unknowns: Does the idea really solve a big customer need? How would this be different from other similar services? Does the organisation have the capability to make it happen?

This corridor meeting could kick of a chain of events that might waste a lot of people’s time, or it could lead to significant growth. It’s the kind of casual growth intercourse that is repeated over and over in corridors everywhere. The colleagues have a choice. They can think, or they can do. Thinking means having another meeting in a week, when nothing has changed, and after an hour’s debate they are still none the wiser. ‘Doing’ on the other hand means they decide, in the corridor to Experiment: “Let’s send a couple of clients hourly arrival countdowns and see what they say, we can do it manually and maybe we could even tell them that next time the service will cost $5 and see if they agree to pay. Let’s do it this afternoon and discuss the idea tomorrow after we see what happens”

OK, there are always a million reasons why we can’t hoodwink our customers, but you get the picture. Experimentation can be pulled out of a pocket and deployed quickly. It replaces dead-end strategy conversation with an active alternative.

Formal and planned

Big businesses need to continuously improve by implementing lots of little ideas, many of them system improvements, and at the same time they need to weed out the innovation pipeline and land a limited number of blockbusters.

There are great tools for seeking out efficiencies and systems improvements like Lean and Kaizen. But occasionally what starts as an obvious ‘just do it’ project can show potential for something much bigger. If you know how to Experiment you are able to spot the potential for gold better than others.

A client of ours once called us to say that their HR team was updating its health and safety advice for colleagues at work and a thought had bubbled up: What if we threw away the volumes of advisory material we produce each month and asked people who have been impacted by a health and safety problem to make a short video of their story, and a reminder of what they should have done differently?

Certainly the upside was huge… but was it a terrible idea or brilliant? The team didn’t want to ‘push it upwards’ as they assumed it would get shot down or laughed at, but deep down they all suspected there was some magic in the idea. We were able to advise them on a programme of Experimentation. Cycle One explored the hypothesis that colleagues didn’t read safety advice.  Cycle Two made a very rough prototype film and assessed the reaction of colleagues. Cycle Three made a better film based on the feedback. All this happened within the team’s decision rights, but quietly. The process was fast and required little additional budget. When the team came to meet with their boss and present a proposal to invest in a more thorough exploration, you can imagine how much richness the Experiment added to their arguments.

On a much larger scale, Experimentation is an appropriate method to incubate new solutions – new products, services or businesses, in live market conditions rather than relying on traditional project management and market research. The online environment is very fertile for Experimentation. We can create prototypes new services and businesses online and within hours have thousands of requests to buy or get involved.  The tools to innovate this way are increasing and improving daily.

Another use of Experimentation is to help shift the behaviour and culture of scores of thousands of employees across many different geographies. We discuss these high impact use cases in the next two articles.

So Experimentation can be applied to identifying the right problem, or accelerating the development of a solution. It can be applied to external customer facing challenges or the issues of internal colleagues. It can be deployed in the corridor, or planned around the Boardroom table. The key for big organisations is to harness its power at scale and go beyond individual teams to create a mass mechanism of Experimentation that’s ‘always on’. This involves becoming what we call an Experiment Engine.

What is an Experiment Engine?

An Experiment Engine is an organisation that has built the corporate muscle of Experimentation at scale.

This gives colleagues the ability to spin up an Experiment quickly, share what they’ve learned effectively and constantly learn from the Experiments of others within the organization.

Becoming a successful Experiment Engine requires successfully managing two things at the same time:

• The ‘Experiment’ part: The skills to design ‘just rigorous enough’ Experiments, a bias for doing over thinking, and the mindset and behaviours that enable teams to constantly ask ‘what if?’
• The ‘Engine’ part: A shared purpose and focus for Experimentation, access to the infrastructure to run Experiments within acceptable risk levels, and ways to share learning across the organisation

Being an Experiment Engine helps to redistribute the responsibility for growth away from the top and spread it more broadly throughout the organisation. When people know the key problems the organisation is committed to solving, and are empowered to create and run Experiments within a defined ‘sandbox’ they build the muscle of learning by trying something new and not just thinking or writing reports. The act of trying, occasionally failing, and trying again builds resilience and most successful innovators will tell you that above all it was their ability to ‘hang on in there’ that underpinned their performance.

All this doesn’t mean that desks are abandoned while everyone is off Experimenting. Since the purpose of an Experiment is to quickly accelerate an initiative or close it down, dud ideas get nipped in the bud whilst those with real promise get more focus.

Think of becoming an Experiment Engine as a giant corporate colonic, flushing out toxic time-wasting projects leaving more time to execute on the projects that matter.

Despite this power and versatility, there are time when Experimentation isn’t appropriate:

• When making a decision is a better option: When solving the challenge is of low importance, or the outcome is quite easy to divine there is little point in Experimenting. The mantra ‘there is no such thing as failure – just learning’ should not be used as an excuse to Experiment around everything.
• Low ability to manage emotion: Experiments tolerate team members’ opposing points of view. It is very instructive to test different rather than same things. But this calls for a great deal of humility. Someone is going to feel ‘wrong’, or worse; embarrassed, exposed, humiliated… The team needs to be fluent in managing emotion.
• Lack of leader support: Experiments work best when at least one person is genuinely backing themselves and believes in the idea or hypothesis. This can be a lonely place to be. The support of a boss during the dark days of Experimentation is critical.

The challenge of retrofitting an engaging ‘human’ process, one that you might imagine happens only in cool tech companies, into legacy-stricken big beasts is not always easy.  Some leaders struggle to inject humanity into a process as serious as ‘strategy’. We think it is the central challenge a growth leader faces and after 25 years on the frontlines of innovation, we’re more passionate than ever about helping supertanker organisations become Experiment Engines.

Want more? Read the second article in this series, Experiment Engines: The Key to Unlocking Cultural Agility.

In the next articles we discuss in more depth how this plays out in practice. If you want to talk more about Experimentation call me now on +44 7970 742 553 or get in touch at matt.kingdon@whatifinnovation.com