What does empirical evidence say about disruptive innovation, and does it matter?

Disruptive innovation has become one of the most popular concepts in business theory. It is fair to say that “disruption”, “disrupted” and “disruptive” have become part of the common vernacular, and it’s common to see it everywhere, about a range of issues, even outside business:

Heck, even my school’s alumni magazine has used the term:

Not surprisingly, it seems that at least a fair few may not truly understand what “disruptive innovation” is. (Of course, I’m not, in bad faith, saying they meant to use “disruption” in the technical and academic sense of the word. This is merely meant to show the popularity of the term.)

Still, the proliferation of this buzzword has led Christian Caryl in The Washington Post to criticise it as “vacuous marketing-speak” and even worse, that it insidiously promotes a “cult” of resignation that robs us of agency (wow). (Opinion: Why the word ‘disruption’ is stupid, lazy — and dangerous, 14 April 2017)

Now, Caryl’s argument is in another domain altogether — for example, he mentions Trump — but here, I hope to evaluate seriously “disruptive innovation" as a business theory.

Disruptive Innovation and Its Usefulness

Published in 1997, The Innovator’s Dilemma by Clayton Christensen sought to explain why well-managed companies still fail — the 4 steps that make up “disruptive innovation” (DI) :

Since then, the theory has taken off. The Economist describes it as “one of the most influential modern business ideas”, applied beyond the business world. And it has abounded in the academic literature:

However, in 2015, King and Baatartogtokh (K&B) conducted a study, whose results are published in the MITSloan Management Review as “How Useful Is The Theory of Disruptive Innovation?”, in which they attempted to test the validity of Christensen’s theory. They claimed, perhaps rightly to some extent, that disruptive innovation is so widely-accepted that not many questioned its predictive power. Its essential validity and generalizability were seldom tested in the academic research; only a few quantitative tests that fail to provide confirmatory evidence.

To do so, they identified 79 experts and conducted a survey of case histories across 77 industries (with questions designed to maintain consistency, and explained as a study of ‘industry transitions’ to prevent bias).

Collectively, these experts reported that (and K&B has several case studies to showcase each point):

• 31% of cases did not involve sustaining innovation.
• 78% of cases did not have incumbents overshooting customer needs.
• 39% of cases did not have incumbents with the capability to respond
• 38% of cases did not have incumbents being disrupted and floundering.

All together, only 9% of cases matched all 4 elements of Christensen’s theory and its predictions.

To explain this, K&B listed some problematic assumptions and other factors at work that challenge the predictive power of disruptive innovation. One is the assumption that sustaining innovation is not enough, but intuitively, better performance is generally desired, even if not exclusively. We all have insatiable needs, and it may not make sense to argue that an offering can ever be too effective, available, convenient, healthy, etc.

Overall, the authors concluded that disruptive innovation is problematic. In light of their findings, K&B suggested that it should primarily serve as a useful reminder of potential pitfalls, such as managerial myopia, making assumptions, and not seeking outside information. It is a warning, not a predictive business theory, and one cannot escape old-fashioned, classical, careful, and fundamental strategic thinking in management. Businesses should use multiple perspectives and not be tempted to seek refuge in a ‘safe solution’ that disruptive innovation could appear to be. It is no substitute for critical thinking and having to make difficult choices and tradeoffs.

Of course, we could critique their research methodology here. Was the number of experts sufficient? What makes them experts? Even K&B acknowledged that their analyses was “open to interpretation” and “may be mistaken”. Was bias really avoided, given the popularity of ‘disruptive innovation’ which may have implicity informed their analysis (such that they perhaps tried to offer an alternative narrative; see the second final thought)? And is this really empirical evidence in the quantitative sense?

A more important question is whether K&B’s analysis goes too far. For them, since less than 10% of the cases explored (another issue of potential selection bias, although the industries surveyed seem to be quite broad) fulfilled all 4 elements of the theory, this implies that the theory is not valid. However, is it nitpicking? We can politely ask whether having all 4 elements is necessary — which are the core ideas of DI and which are the peripherial ones? In addition, DI is more than that; its 5 principles, for one (e.g. an organization’s capabilities define its disabilities; technology supply may not equal market demand) — although one can, in turn, question whether they are too broad to be right but not useful.

What is a business framework (for)?: Explanatory versus normative purposes

In 2015, Christensen (and others) co-wrote an article in Harvard Business Review, “What is Disruptive Innovation” in which he clarified several points regarding disruptive innovation:

• It will not apply “to every company in a shifting market”.
• It needs to be continually developed, to “integrate insights from subsequent research and experience into the original theory”.

Most strikingly, they stress the importance of maintaining the theory’s “usefulness”, even whilst he acknowledges that disruptive innovation “does not, and never will, explain everything about innovation specifically or business success generally”. Indeed, he argues that “integrating [the many other forces and factors] into a comprehensive theory of business success is… unlikely… anytime soon”.

This raises an interesting point about disruptive innovation used as a tool, instead of a theory. In “The Decision Book: Fifty Models for Strategic Thinking”, the authors Mikael Krogerus and Roman Tschappeler identified the following as characteristics of a decision-making model:

• They simplify: they do not embrace every aspect of reality, but only include those aspects that seem relevant.
• They are pragmatic: they focus on what is useful.
• They sum up: they are executive summaries of complex interrelations.
• They are methods: they do not provide answers, they ask questions; answers emerge once you have used the models, i.e. filled them out and worked with them.

If so, then perhaps disruptive innovation should not (or could not) be considered as an empirical theory which seeks to explain the nature of (some) innovation, but rather, simply a model for managerial practice.

The case of the iPhone

Indeed, when Christensen was asked, during an interview in 2020, shortly before he passed, if he got anything wrong in hindsight, he mentioned his famous mis-prediction of the Apple iPhone:

The prediction of the theory would be that Apple won’t succeed with the iPhone … History speaks pretty loudly on that.

But of course, history went down a markedly different path.

Still, Christensen had a response:

One of my former students, Horace Dediu, taught me that I had framed the problem incorrectly. I viewed Apple as a late entrant into the mobile phone business, where in Horace’s view it was an early mover in the “computer in your pocket” business… This example reinforced to me the importance of getting the categories right. When someone tells me they are disruptive, the first question I always ask is, “To what?” This is an important question, because disruption is a relative concept.

This may seem to vindicate disruptive innovation, but it raises a more fundamental question. If we can always look back and adjust the definitions, context and objects to which the theory is applied, how useful is that theory, really? Indeed, any philosophy of science student would know the famous criterion set forth by Karl Popper for a theory to be considered ‘scientific’ (or social scientific for our purposes): falsifiability by prediction.

A theory, to be scientific, needs to also be able to provide for measurable predictions, then it could be conclusively disproven, and that risk makes a theory more rigorous and robust. If, however, instead, we can always find ‘evidence’ to support a theory, and interpret the past differently to confirm the theory was valid ‘all along’, then it may not even be useful. As such, the important question, especially for managers, is whether disruptive innovation can actually make accurate predictions.

In a review of the theory in the Journal of Management Studies, Christensen et al. (2018) recognized this key criticism:

One particularly salient issue concerns whether disruption is a concept that can only be experienced after the fact. That is, does it allow for ex‐ante prediction (rather than just ex‐post explanation) about whether a particular innovation will eventually challenge leading incumbents (Christensen, 2006; Danneels, 2006; Markides, 2006)? Indeed, theories aim for prescriptive implications; they provide useful advice to individuals and organizations (Bazerman, 2005).

In response, the authors argued that DI should be valued as a useful tool, rather than as an explanatory or predictive theory. They list “several publicly documented cases of how companies facing disruptive threats used the model to achieve growth and market leadership”, such as Amazon’s Kindle business (Stone, 2013, pp. 233– 237), The New York Times (Benton, 2014), and Wealthfront (Rachleff, 2013), which had referenced how DI theory informed their respective innovation strategies (which supposedly turned out ‘successful’).

As such, the authors conceded that DI may not be right always, but implied that it does better than chance and is thus useful. They cite further examples: a 2004 study, Christensen et al. used DI to predict ex‐ante outcomes in different industries; outcomes later observed were consistent with predictions in four of the six industries (Christensen et al., 2004). Raynor (2011b) compiled data on 48 ventures launched as part of Intel’s internal corporate venturing program; blind to actual outcomes, they developed hypotheses based on DI: specifically, if an innovation was sustaining and Intel was an incumbent in the target market, the venture would succeed (fail); if the innovation was disruptive and an autonomous business unit was formed to pursue it, the venture would succeed (fail). Using business plans to classify the ventures and survival (demise) to proxy performance, DI theory had a statistically significant impact on correctly predicting the outcomes of the businesses (Raynor, 2011a).

In Christensen et al. (2018)’s words:

Indeed, [business/management] theories aim for prescriptive implications; they provide useful advice to individuals and organizations… But together with other empirical evidence and the specification of a causal mechanism, these studies provide intriguing insight for a normative theory of disruptive innovation… We have charted how a descriptive account of technology change evolved into a normative theory of innovation and competitive response”

If so, seems we have reached a potential middle ground. Yes, empirically and rigorously, DI theory may not be a valid explanatory theory that explains or predicts most business cases, but it could still be useful as a managerial tool.

But even purported usefulness could have its own limits.

The case of Blue Ocean Strategy (BOS)

Introducing another theory might be helpful here. And that is blue ocean strategy (BOS), proposed by Kim and Mauborgne, and the “value innovation” it advocates. BOS is also a popular and very intuitive theory, and similarly, proponents typically claim that BOS is a universally applicable concept that can potentially lead to great performance improvements for all types of organizations.

The evidence offered by them, however, is also mostly case-based and anecdotal in nature (e.g. interviews on the BOS webpage), and there are relatively few studies which have looked at the performance effects of implementing the concept. In other words, similar to DI theory, there seems to be a lack of empirical evidence for BOS (Parvinen et al., 2011; Madsen & Slatten, 2019).

Could these 2 supposedly universal and intuitive theories be right? Probably not. While there are some similarities, their core focus is different. As Hammer (2022) describes,

Creative Destruction or Disruptive Innovation occurs when an innovation (often technologically based) breaks up and conquers an existing market by displacing an earlier technology, product or service. The word “displacing” is particularly important in this case, since without “displacing” out there can be no “destruction” (of the existing market and competitors)… the old is constantly being destroyed or replaced by the new. In contrast to creative destruction, the Blue Ocean Strategy does not require displacement or even destruction. The Blue Ocean Strategy… that goes beyond creative destruction and has its overarching focus on creation.

In Kim and Mauborgne’s own words, in response to the question, “Is Blue Ocean Strategy synonymous with creative destruction (a similar, earlier concept by Schumpeter with many similarities with DI):

No. Blue ocean strategy goes beyond creative destruction to embrace nondestructive creation. Creative destruction occurs when an innovation disrupts an existing market by displacing an earlier technology or existing product or service… Blue Ocean Strategy, in contrast, does not necessitate displacement or destruction.

(This also begs a more ‘meta’ question: could there even be a single universal theory of innovation, given innovation is the creation of something new, which perhaps explains the retrospective way such theories are developed?)

More importantly, in terms of managerial implications, since both theories are fundamentally, to some extent, incompatible, they can’t both be right all the time. If so, then we must identify the boundary conditions wherein either is valid and can be applied. And that requires using some objective metric — i.e. empirical evidence — to differentiate, since mere intuition or reinterpretation can work for both.

In other words, potential, intuitive usefulness for managers alone may not be enough, since they must then decide if and when to use one theory or framework over another (e.g. DI vs. BOS vs. something else).

And that is a problem for a merely useful theory — there is more than one theory that is useful. For managers who face different choices, each with its own opportunity cost, having an explanatory and predictive theory, or at least understanding the boundary conditions within and without which it works, to some reasonable extent, is critical.

Some final thoughts

Management fashion theory

It is arguable that, as the preamble has implied, business education is awash in a sea of buzzwords.

Management fashion theory has arisen to explain why some theories become popular and diffuse versus others which don’t, or why a theory’s popularity rises and falls (Abrahamson, 1991, 1996; Kieser, 1997; Newell, Robertson & Swan, 2001). The key idea is that there is a ‘market’ for management theories, and thus, the popularity thereof depends on, and is shaped by, supply-side and demand-side factors. On the demand-side, some theories become popular because they have certain characteristics that make it appealing to organizations and managers, and are a good fit with the zeitgeist of the times. Perhaps more importantly, on the supply-side, there is a “fashion-setting community”, to use Abrahamson (1996)’s words, comprised of consulting firms, management gurus, etc., that work to propagate (or dismiss) a theory.

That said, these supply-side actors do not simply force fashions onto businesses and managers. They need to use “rhetorics”, competitively against others, that react to a sensing of the “emergent collective preferences of managers for new management techniques”. As such, perhaps this rhetoric, in the form of catchy words and a compelling narrative, may make empirical analysis of the theory difficult and reduce its predictive power.

The transformative power of business theories

Furthermore, a theory’s popularity could also affect its predictive power. Theories, especially social scientific theories, do not operate on merely passive actors (managers, customers, and business organizations). Carton (2020) argues that the actors, artefacts, or practices that are intertwined with, and co-produce, theories (which he terms “assemblages”) are changed by, and change, the theory itself, such that a theory can change reality, which then enlarges its scope in turn. In other words, a theory that purportedly explains or predicts reality, once disseminated and popularized, can be made into reality. Similarly, once a theory is known, those supposedly affected may work to prevent the predicted outcome. This begs the question about whether business theories could, to some extent, be self-fulfillng/defeating.

Endnote

This topic was done as a paper presentation of K&B’s article in an seminar-style course I took at the National University of Singapore, BSN4811 Innovation and Productivity, taught by Prof Ivan Png. As a typical business student, I had answered the following to the title question which was tasked to me:

1. There are two perspectives regarding a business theory — explanatory vs. predictive / normative. For managers (versus academics), its normative value, as a decision-making tool, is more important, especially as innovation is forward-looking, even if it is not consistent with empirical evidence.

2. There will always be special cases and counterexamples to any business theory, which is based on assumptions and simplifications of complex reality. The more a business theory aims to be universally-applicable/explainable, it risks losing its core idea and the value-adding perspective that helps managers analyse, understand and strategise. (To spread its influence, a theory requires a certain packaging. When used explanatorily, not all aspects of that packaging could be met by reality.)

3. Ultimately, we can never fully know whether DI really helped, we applied DI correctly, or it is just a coincidence or misapplication when it matches / does not match with empirical evidence. Every business case now is both different and similar to past cases. Still, there’s no harm in trying, carefully and thoughtfully, DI, along with and other analysis and strategy tools!

In short, to answer the question: “For the purposes of helping managers, does it matter whether Clayton Christensen’s theory of “disruptive innovation” is consistent with the empirical evidence?”

Not really. What matters is the potential usefulness of the core ideas and the value of adopting the general perspective, for the manager’s specific business problem or innovation strategy. Even if its explanatory power would fail sometimes, and not all aspects consistent, managers should at least consider a theory which has analytical and normative value.

But after the spirited discussion with my professor and classmates, I have come to adopt a different position, which I have written above, but can be summed up by my professor’s simple comment, “That’s all well and good, but then how do you know it’s right (will be useful)?”

Yes, how do you know whether and which one you should use? Rigorous empirical evidence is needed to validate our intuition.

There could be real harm in just trying.