Editorials

How will we know if the London 2012 Olympics and Paralympics benefit health?

BMJ 2010; 340 doi: http://dx.doi.org/10.1136/bmj.c2202 (Published 20 May 2010) Cite this as: BMJ 2010;340:c2202
  1. Mike Weed, professor of sport in society
  1. 1Centre for Sport, Physical Education and Activity Research (SPEAR), Canterbury Christ Church University, Canterbury, Kent CT1 1QU
  1. mike.weed{at}canterbury.ac.uk

    By measuring directly attributable effects in addition to opportunity costs

    The London 2012 Olympic and Paralympic Games will cost £9.3bn (€10.7bn; $13.3bn), £150 for every man, woman, and child in the United Kingdom. For this investment, we have been promised legacy outcomes1 for sport and physical activity, regeneration, culture, sustainability, the economy, and disability. The last of these legacy outcomes was added only recently,2 after considerable criticism.3 Each of these areas has implications for health or relates to socioeconomic determinants of health.4

    In the linked systematic review (doi:10.1136/bmj.c2369), McCartney and colleagues found little evidence that major multi-sports events deliver health or socioeconomic benefits.5 This suggests that £150 a head towards staging London 2012 is a poor investment made by the treasury on our behalf. However, the review shows that past research comprises a small number of poor quality studies, with large gaps in the outcomes evaluated. Furthermore, studies have evaluated incidental outcomes; London 2012 is the first Olympic and Paralympic Games that will explicitly try to develop socioeconomic legacies for which success indicators are identified—the highest profile of which is to get two million more people more active by 2012.1 London 2012 therefore seems to fulfil one of McCartney and colleagues’ recommendations to include longer term outcomes as legacy goals,5 and, in the case of increased physical activity, the goal is explicitly linked to improved public health.6 7

    However, if the chosen measure for this legacy outcome, the active people survey,8 shows that two million more people are more active by 2012, it will not be an indication that London 2012 has increased physical activity levels. This is because the survey cannot demonstrate attribution, which put simply means that it cannot provide evidence that London 2012 intervention programmes are the cause of two million more people becoming more active. Neither can the survey provide evidence of “additionality,” which means it cannot show that London 2012 programmes have increased physical activity to levels greater than could have been achieved by investment in alternative interventions. McCartney and colleagues’ review highlights these difficulties, with no included studies able to attribute changes to events, and few containing contemporary comparisons against which opportunity costs can be considered.

    So, how will we know if our investment of £150 a person has generated additional health and socioeconomic outcomes clearly attributable to London 2012? Evaluation must focus on London 2012 intervention programmes rather than generic national surveys and consider net outputs not gross positive indicators. Free swimming for under 16 year olds and over 60s (a London 2012 intervention programme) may show high take-up by the public.9 However, this may be confounded by several factors such as people who already swim simply doing so free of charge or increasing their frequency of swimming. In this case, free swimming would not have had additional benefit because it would not have resulted in more people being more active, and evaluations must take care to remove such behaviours from impact calculations.10

    Opportunity costs must be considered if outcomes are to be attributed to London 2012. Because each major multi-sports event is unique, the contemporary comparisons that McCartney and colleagues call for cannot be derived from control groups or reference cases.11 Detailed alternative scenarios, termed “counterfactuals,” outlining what would have been most likely to happen in the absence of London 2012, must be modelled for comparison purposes.10 11 Unfortunately, the London 2012 legacy evaluation framework develops examples of only the most basic counterfactuals.12 In fact, in two of four examples given, the framework outlines alternative scenarios in which there would be no alternative activity, indicating that there would be no opportunity cost. This suggests a limited understanding of how the principles of attribution and of additionality must be applied in practice.

    Evidence provided in a systematic review for the Department of Health leads to the conclusion that detailed modelling of counterfactuals requires alternatives to be outlined in at least three broad areas.9 Firstly, a consideration of economic alternatives must model what would have happened to funding for intervention programmes without London 2012—would some or all of it remain within the relevant sector? Secondly, a consideration of alternative themes must outline what promotional messages would have been used as hooks to engage people in place of Olympic or Paralympic themes. Finally, alternative scenarios for support and enthusiasm must consider what political, practitioner, private sector, or third sector (or combinations thereof) support and enthusiasm would exist for intervention programmes without London 2012. It is unlikely that free swimming would exist without London 2012, so the question is whether the funding would have been lost to the sector, or whether it would have been invested in other physical activity programmes, and if so for whom, with what emphasis, and with what levels of support and enthusiasm. Once these questions have been answered and an alternative scenario to free swimming established, its likely effects can be modelled from previous research. This provides a comparator case against which the effectiveness and cost effectiveness of free swimming can be measured, along with its true attributable and additional contribution to the physical activity legacy of 2012.

    McCartney and colleagues conclude that how the costs of major multi-sports events can be justified in terms of health and socioeconomic benefits is unclear.5 Given that the legacy evaluation framework provides only a basic outline of how to apply in practice the concepts of attribution and of additionality to the opportunity costs,12 the risk for the UK population is not that we will not get the benefits we want for our £150 a head investment in London 2012, but that there will be no robust evidence of what we have paid for.

    Notes

    Cite this as: BMJ 2010;340:c2202

    Footnotes

    • Research, doi:10.1136/bmj.c2369
    • Competing interests: The author has completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declares: (1) No financial support for the submitted work; (2) MW received funding from the Department of Health to complete a systematic review of the evidence to underpin physical activity, sport, and health legacies from the London 2012 Olympic and Paralympic Games that was published in 2009; (3) No spouse, partner, or children with relationships with commercial entities that might have an interest in the submitted work; (4) No non-financial interests that may be relevant to the submitted.

    • Provenance and peer review: Commissioned; not externally peer reviewed.

    References