What football teaches us about researching complex health interventionsBMJ 2012; 345 doi: https://doi.org/10.1136/bmj.e8316 (Published 16 December 2012) Cite this as: BMJ 2012;345:e8316
- Alexander M Clark, professor 1,
- Thomas G Briffa, research associate professor2,
- Lorraine Thirsk, assistant professor3,
- Lis Neubeck, senior research fellow4,
- Julie Redfern, senior research fellow4
- 1Faculty of Nursing, Clinical Sciences Building, University of Alberta, Edmonton, AB, Canada T6G1C9
- 2School of Population Health, University of Western Australia, Crawley, WA, Australia
- 3Faculty of Nursing, University of Alberta, AB, Canada
- 4George Institute for Global Health, University of Sydney, Sydney, NSW, Australia
- Correspondence to: A M Clark
Who would you rather have as a player on your football team: Messi or Clark⇓? Both players share numerous characteristics, such as they both have brown hair, have the same size feet, and are less than 6 ft (1.8 m) tall. Each has scored many goals, playing in the number 10 jersey.
However, focusing on these overt characteristics is not a good basis for decision making. Close observation, informed assessment, and knowing the context of previous successes (goals against whom and on what occasion) provide more useful insights into the determinants of success in football. Lionel Messi, the Argentinean international professional player, is infinitely preferable to Alex Clark, an amateur from the University of Alberta, Canada. Yet research into complex healthcare interventions still focuses on easily described components of interventions and risks overlooking what really matters.
Complex versus complicated
Interventions in football and healthcare systems are “complex” rather than “complicated.”1 Phenomena are complicated when intervention outcomes can be reliably predicted from past behaviour with the help of mathematical analysis. Sending a rocket to the moon is complicated.2 However, phenomena are complex when too many factors are interacting. In such situations formulas have limited application and similar past experience is a poor predictor of future success.2 Raising a child is complex—doing the same things at different times often results in quite different outcomes.2 Accordingly, in football, formula driven approaches have consistently failed,3 and a health intervention that succeeds in one setting may have very different results in another.
Complex interventions in football and healthcare have a range of shorter term and longer term outcomes (table 1⇓) and are composed of many components that are made up of smaller subcomponents (table 2⇓). Outcomes are generated by dynamic interactions between these components, not only with each other, but also with aspects of context and a wide range of other potentially influential laws, variations, and unpredictable factors (table 3⇓).
Because of this complexity, outcomes in football and healthcare are not chaotic (random over time) or uniform (identical over time). Rather, outcomes are somewhat patterned. Some football players successfully complete passes more often than others, and identical medical interventions can result in very different outcomes in different doctors’ hands. But unexpected outcomes still occur. Messi still misses chances he should score from, and an intervention to promote diabetes self care that was effective in one setting,4 and is supported by meta-analyses,5 may not have benefits in another setting. Given their shared complexity, we suggest some lessons that healthcare research can learn from football.
Lesson 1: Ontology—bring complexity in
Because football and healthcare are complex, describing interventions and explaining their effects requires attention to ontology: the underlying ways in which interventions are understood.5 Football and its discourses reflect many aspects of complexity. Outcomes can be influenced by individual components (a manager), subcomponents (a single player’s attitude), context (a muddy pitch), and a range of uncontrollable factors (injury to key player). Deeper still, interactions between these elements may occur and generate new effects—for example, the gifted player who underperforms in the context of the “big match” with a hostile crowd.
In healthcare interventions, ontology seems to be thought of as irrelevant or a luxury when compared with the attention given to methods, measurements, and results.5 6 Yet ontology shapes not only these aspects but also the questions that research should and can ask. Asking the question “Does this self-care intervention work?” risks adopting the flawed but common assumption that it is only the intervention that determines effectiveness, irrespective of time, place, and context.7 This is akin to asking “Does this football team win?”—it assumes wrongly that a team can and will win every time.
More sophisticated methods are often incorrectly seen as an adequate substitute for ontology.8 The failure of econometrics (arguably the most sophisticated quantitative discipline handling “big data”9) to predict the global recession illustrates this error.8 9 Research into healthcare interventions should measure outcomes well and use appropriate methods, but it has to be based on ontologies that adequately reflect complexity.5 6 10
Lesson 2: Clarity—describe interventions well
Discussions about football tend to take account of many large parts of games (such as the presence or absence of particular players, teams, referees, and managers) as well as smaller parts (such as these people’s skills, characteristics, experience, and tendencies). Conversely, comprehensive descriptions of the many components of healthcare interventions are mostly absent from publications.10 11 Multifaceted interventions are often handled methodologically as single agents.5 Components that are selected for more detailed description and incorporated into analysis tend to be those that are more easily quantifiable or physical in nature,6 such as an intervention’s duration or means of delivery. However, as with Messi, this risks missing the most powerful drivers of effectiveness—which may be less quantifiable but potentially more influential—such as the skills, experience, and values of those providing the intervention.4
The components of healthcare interventions should be described in research.6 Taxonomies that describe interventions comprehensively and systematically are needed. Components that theory, observation, and other data suggest may contribute more to changes in outcomes should be included in these descriptions.
Lesson 3: Why?—don’t just describe outcomes, explain them
Outcomes, on their own, tell little of what has generated them. Results are likely to be improved only when we understand what has contributed to past outcomes. Discussions in football consistently seek explanations for what has generated outcomes, such as the presence of a particular player in the team or the qualities of a particular player (“Clark can’t run or shoot properly”). Suggestions abound as to what could or should be done to increase the probability of a more favourable outcome next time.
By contrast, attempts to explain outcomes of healthcare interventions by “opening the black box” are still relatively rare,5 12 and they are dominated by an over-riding focus on results, especially when findings are favourable and statistically significant.5 A randomised trial can show whether a patient counselling intervention worked but not why or how it worked.4 A meta-analysis can aggregate the results of trials of sufficiently similar counselling interventions over a set period of time.5 Sensitivity analysis or meta-regression can identify what components of these interventions contributed most to results, but this depends on underlying trials being well described, which is seldom the case.11 As such, meta-analyses usually provide a measure of general trends in results but do not explain these trends. In football terms, this equates to simply aggregating all past results against sufficiently similar teams or the same team over a set period of time.
Explanation matters. Its ongoing relative absence from research into healthcare interventions reduces the capacity of research to improve outcomes. More research and theory are needed to identify which components of healthcare interventions have more influence on outcomes and why. Outcomes from interventions should be measured, but studies should also incorporate different qualitative and quantitative techniques to better explain these outcomes.12
Lesson 4: Opportunity—learn from failure and success
“Bad” results in healthcare and football usually negatively affect emotions, perceived status, reputation, power, and identity.
In football, bad results tend to lead to greater attempts to explain and improve outcomes.13 Contributing factors are often seen to reside in components (manager’s poor tactics) or subcomponents (fatigue of a skilful player), contextual interactions (such as negative effects on team morale of past bad results), or uncontrollable factors (notably seemingly “biased” referees).
Conversely, in healthcare research, failure is often presented as success: the results of 40% of studies with negative findings are “spun” into positive results,14 or even turned into false “wins” through questionable adjustments, such as stopping data collection early or excluding outlying data.15 But how will outcomes be improved if the opportunities gifted by failure are not harnessed more fully? It is important to learn both from what works and what does not work.4 Failure to attain successful outcomes in healthcare interventions can generate especially useful lessons for intervention refinement.
Study designs should be used that harness these lessons for future interventions.
What can we learn?
Football illustrates the folly of ignoring complexity. Healthcare researchers can learn from football by describing the important components of interventions more comprehensively and, irrespective of results, using research approaches that take the complexity of interventions into account and seek to explain outcomes better. Such an approach would not only improve the quality of research into healthcare interventions but also increase its uptake by practitioners and its ability to improve outcomes in clinical practice.16
That said, football can be criticised for being unscientific. Prejudices for and against players and teams can cloud judgment. Emotional over-involvement, anecdotal post hoc rationalisation, and centralism (the tendency to explain outcomes by a small number of individual factors) are common.17 However, philosophers of science over the past 50 years have suggested that scientists—and their discussions, processes, and findings—are also prone to strikingly similar personal leanings, group tendencies, and vested interests.18 19 Attempts to understand and improve outcomes in both healthcare and football are best strengthened not only by harnessing data, but also by reflexivity, transparency over conflicts of interests, and genuinely open minded and informed dialogue, particularly with those who hold different views.20
Like football, healthcare is a complex adaptive system in which interventions are also complex
Healthcare researchers can learn from football about describing the important components of interventions more comprehensively
Approaches that take the complexity of interventions into account could help explain outcomes better so that more can be learnt from failure
Taking complexity of healthcare research into account would improve the quality, usefulness, and translation of research into practice
Cite this as: BMJ 2012;345:e8316
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review: Commissioned; externally peer reviewed.