UK alcohol industry’s “billion units pledge”: interim evaluation flawedBMJ 2015; 350 doi: https://doi.org/10.1136/bmj.h1301 (Published 24 March 2015) Cite this as: BMJ 2015;350:h1301
- John Holmes, senior research fellow,
- Colin Angus, research associate,
- Petra S Meier, professor of public health
- 1Sheffield Alcohol Research Group, School of Health and Related Research, University of Sheffield, Sheffield S1 4DA, UK
- Correspondence to: C Angus
- Accepted 17 February 2015
The UK government’s 2012 alcohol strategy proposed several evidence based policies, including minimum unit pricing.1 However, few of these policies have been implemented. Instead, the government has increasingly pointed to the industry’s pledge to remove one billion units from the alcohol market as an indication of what can be achieved through voluntary deals with industry.2 For example, the billion unit pledge was the only alcohol intervention mentioned listed as deliverable in the Department of Health report Living Well for Longer in July 2014.3 Although the latest interim analysis claims that the pledge has been met,4 we believe the data used in the analysis may not be fit for purpose, that the report makes simplistic assumptions about consumer responses to the pledge, and takes insufficient notice of confounding factors.
Billion unit pledge
The pledge by 33 alcohol producers, retailers, and trade organisations began in 2012 and was announced as part of the Public Health Responsibility Deal, the government’s flagship public health policy.5 Under the responsibility deal, industry partners pledge to undertake activities that ostensibly address public health problems. The government argues that such voluntary agreements are preferable to costly and cumbersome regulatory approaches. The billion unit pledge committed signatories to “remove one billion units of alcohol sold annually from the market by December 2015, principally through improving consumer choice of lower alcohol products.”6
In practice, this means that the alcoholic strength of some beverages will be reduced and new beverages branded as low or no alcohol will be developed and promoted by industry. However, the evidence base for achieving public health gains by promoting reduced strength alcohol is limited.7 In theory, if consumers simply replace higher strength products with lower strength alternatives, overall alcohol consumption and associated harm will fall. But it is also possible that consumers will ignore the new products or drink them in addition to existing products, increasing the total amount of alcohol being consumed.
This potential for higher consumption prompts legitimate questions regarding industry motives for focusing on producing lower strength drinks. Consumer tastes are reported to have shifted away from high strength products and brewers are also keen to attract women, who are perceived to prefer lower alcohol drinks.8 9 The desire for lower strength drinks was acknowledged in the press release announcing the deal10 and raises the question whether new low strength products are primarily designed to reduce alcohol consumption, simply reflect what consumers want, or seek to increase consumption in new drinkers. Is manufacturer motivation driven by public health concern or commerce?
Problems with latest interim evaluation
The responsibility deal website says that the industry pledge will be judged a success if the annual number of units of alcohol sold across the UK is reduced by at least one billion (roughly equivalent to a 2% reduction from 2011 levels). It also says that the reduction must be shown to result from the pledge.11 This second aspect is the more challenging to demonstrate.
In December 2014, the Department of Health produced its second interim report on progress towards meeting the pledge. It concluded that 1.3 billion units had been removed from the market as a result of the pledge between 2011 and 2013, exceeding the target two years early.4 However, a closer look at the analyses and data that underpin this headline figure raises questions, as we describe below. It is therefore unclear how much of the recent changes in alcohol consumption are truly attributable to the pledge. Further details of our analysis are available in the technical appendix on thebmj.com.
Data may be unfit for purpose
The health department’s interim evaluation draws on two data sources. The first is HM Revenue and Customs (HMRC) excise duty (alcohol tax) data, which show how much alcohol is sold in the UK in any given year.12 Beer and spirits are taxed relative to their alcohol content, making this a reasonable source of information for quantifying sales of alcohol within these drinks. However, wine and ciders are taxed by volume of product irrespective of strength.13 This means tax data do not provide information on the number of alcohol units in the market for wine and cider.
The report therefore used additional market research data on alcohol retail sales in Great Britain to estimate quantities of wine and cider sold (table 1⇓).14 But the estimated volume of alcoholic drink sold in these data is 9% lower than the equivalent estimate from HMRC in 2011 and 14% lower in 2013. Reasons for this include limited information on discount retailers and the exclusion of Northern Ireland.4 14 The increase in the discrepancy over time is important because it is larger than the claimed effect of the pledge—a 1.3 billion (or 2.6%) reduction in units sold. A proportion of this effect could therefore be due to the changing discrepancy between data sources.
In the appendix we also present a case that 870 million of the 1.3 billion units removed from the market are accounted for by a change in 2011 to the way HMRC recorded beer data, rather than the pledge. As the government evaluation did not take steps to address these data problems, we conclude that the estimated effect of the pledge may be invalid irrespective of our other concerns.
Only one consumer response is considered
The government analysis assumes that people will respond to lower strength drinks by consuming the same amount of beer or wine but at a lower strength (a lower percentage alcohol by volume or abv). However, we believe that this simplistic approach is flawed.
The report uses three outcome measures: the total annual volume of a beverage sold; the percentage of that volume which is pure alcohol (abv), and the number of alcohol units sold (1 unit=10 ml or 8 g of pure alcohol). These outcomes are obtained for five categories of beverage (beer, cider, wine, spirits, and premixed spirits or “alcopops”) and values for 2011 and 2013 are compared using the following equation (table 2⇓):
Change in units sold=(2013 abv×2011 volume of beverage)−(2011 abv×2011 volume of beverage).
However, this equation is not valid if, for example, people drink lower strength products when they would have previously chosen a non-alcoholic drink. Although in this case consumption would have increased, the lower average abv could result in the calculation showing a decrease (see technical appendix for details of how the results are affected by different consumer response scenarios to the one assumed by the evaluation).
Change in the analysis plan
The analytical approach was altered after public health stakeholders resigned from the responsibility deal alcohol monitoring and evaluation group. The change meant that the headline measure of the pledge’s success was based on analysing all beverages separately then adding up the results, rather than analysing all beverages together (Mark Bellis, personal communication). The report does not acknowledge this change but argues that use of a beverage specific analysis controls for underlying trends and confounding in the market share of each beverage type:
“Some of the change in the total volume of pure alcohol that occurs from year to year is due to changes in the category shares. For example, as the average abv of beer is lower than the average abv of wine, a shift in market share from beer to wine would (other things being equal) result in an increase in the total volume of pure alcohol and an increase in the overall average abv. By holding constant the volume of products the change in the number of units driven solely by changes in abv can be calculated.”4
This methodological change is important. The first interim report found 253 million units had been removed from the market6; however, had the original method been used, it would have concluded that the pledge led to 359 million units being added to the market. In the second report analysis by beverage type made little difference.
Confounders inadequately considered
The only consideration given in the report to potential sources of confounding is to the market share of each beverage type. However, three other important sources of confounding are not controlled for.
Firstly, the government introduced increased taxes on high strength beer and reduced taxes on low strength beer in 2011. Lower taxes on low strength beer mean these products can be sold more cheaply or more profit can be made if they are sold at the same price as other beers. Given this incentive to producers and consumers, sales of these products are likely to have increased irrespective of the pledge.
Secondly, the estimated annual number of units of alcohol sold in the UK fell by 18% between 2004-05 and 2012-13.15 This trend has not been uniform across the population or across types of alcoholic beverage. For example, there have been long term shifts in consumption from beer to wine, and from drinking by younger people to drinking by older people. These shifts affect the volume of each beverage type sold over time and the total amount of alcohol sold.16 17
Finally, the average strength of products has not been stable over time; the average alcohol content of beer sold fell from 4.57% in 2000 to 4.48% in 2010.16 Robustly separating the effect of the pledge from these factors requires a careful and detailed statistical analysis which the report does not provide.
The above critique does not imply the billion unit pledge is bad for public health, simply that it may be too challenging to evaluate whether the target has been met. The evaluation plan stipulated that any reduction in alcohol sales must be shown to result from the pledge. Until these problems of data, plausible consumer responses, and confounding (table 3⇓) have been addressed we question the validity of the conclusion that 1.3 billion units have been removed from the market as a result of the pledge alone.
It is not clear whether a robust quantitative evaluation of the billion unit pledge is possible with currently available data. The ideal would be to have individual level longitudinal data on alcohol consumption with sufficient detail to allow analysis of changes in the products consumers purchase and whether new lower strength products substitute for or add to existing consumption. To our knowledge, such data have not been collected.
In the absence of these data what are the alternatives? We believe that greater onus should be placed on process evaluations that assess whether producers’ behaviour and products have changed in a way and to a scale likely to facilitate healthier drinking habits. This may include, for example, reporting on what products have been introduced or altered in strength, with accompanying abvs, and supplying sales data for these products. Currently provision of such data is voluntary, and many signatories make more generalised claims. For example, Bacardi Brown-Forman (which makes Bacardi rum, Jack Daniel’s whiskey, and Southern Comfort) states in its progress report: “The ABV on some products has been reduced and new products at a lower ABV than the parents’ brands have been introduced.”18
Again, although it is unlikely to quantify progress towards the pledge, independent market research could also help understand how consumers are responding to new and revised products, in particular who is buying them and whether they are substituting for or adding to existing drinking.
We recommend the billion unit target is abandoned in favour of measurable alternatives. We also recommend that the Department of Health withdraws the 2014 interim report, requests stakeholders not to cite its conclusions, and reviews the evaluation approach.
The responsibility deal includes a pledge from the alcohol industry to remove one billion units from the market and one of the UK’s central alcohol policies
The Department of Health’s interim evaluation of its success makes questionable assumptions and ignores key confounding factors
A robust quantitative evaluation of the effect of the pledge may not possible
The billion unit target should be dropped in favour of measurable alternatives
Cite this as: BMJ 2015;350:h1301
We thank Katherine Brown of the Institute for Alcohol Studies for comments.
Contributors and sources: JH, CA, and PM are public health researchers specialising in quantitative appraisal of alcohol policy options. All authors contributed to drafting of the manuscript. Additionally, JH conceived and was the lead –author of the manuscript and is guarantor of the article, CA and JH developed the critique of the interim report, and PM oversaw the work.
Competing interests: We have read and understood BMJ policy on declaration of interests and declare the following interests: PM is a member of the scientific advisory board of the Institute for Alcohol Studies, a non-profit organisation advocating for the use of scientific evidence in policy making to reduce alcohol related harm.
Provenance and peer review: Not commissioned; externally peer reviewed.