Re: Investigating the relationship between quality of primary care and premature mortality in England: a spatial whole-population study
We appreciate the comments of all the correspondents who took time to read and respond to our paper. Before addressing the tangible points raised, we want to make it clear what we did not say in the paper: that primary care does not save lives; that the investment made in the QOF was squandered; that the effort made by thousands of practice staff in meeting QOF targets was wasted; nor that the QOF has had no impact on patient care and outcomes. Furthermore, we cited evidence on the value of primary care in reducing premature mortality, and we have previously contributed evidence on positive impacts of the QOF.
Improving the management of common chronic conditions has been a worthwhile achievement by practices, but it does not necessarily translate into longer lives for patients. In our latest paper, we therefore asked whether there was any association between practices’ performance on the QOF and mortality rates in the populations they serve. Some of the respondents may have preferred us to ask slightly different questions.
We agree with Honeyford et al that more in-depth investigation of specific pathways of care is warranted, however our aim was to investigate whether there is any association between overall QOF performance and mortality. This is an important policy question. Our third performance metric did address intermediate outcomes related to deaths from specific conditions, but even using this measure we could find no association with mortality. On the correspondents’ other points: we did not use recorded prevalence as a measure of primary care quality because this is not directly incentivised under the QOF, and we took the alternative view that register sizes depend on ‘true’ prevalence in the community (which in turn is partly dependent on survival), not just on completeness of case finding. We would also argue that a better measure of practice performance on hypertension screening might be the relevant public health/organisational indicator: the proportion of patients aged over 40/45 with a blood pressure recording. With respect to smoking, although the correspondents have produced a robust method for estimating practice-level smoking rates from QOF indicators [1], it is only applicable from 2012/13, when the relevant indicators were introduced. However, the mortality risk from tobacco use is mediated primarily through cancer, cardiovascular and respiratory disease: we included denominators for these conditions in our models, in addition to area deprivation, which is also strongly correlated with smoking rates.
We were not too surprised that our findings at the LSOA level were not consistent with those of Honeyford et al at the PCT level. PCT-level analyses aggregate heterogeneous areas with respect to the strongest predictors of mortality, such as area deprivation, and we took a different approach to key covariates: for example, our models included rurality and continuous (rather than binary) measures of age. However, we note that only one of the QOF quality indicators modelled in Honeyford et al’s 2013 study reached borderline significance [2], and their 2010 study concluded that “neither provision of primary health care… nor clinical performance as reflected by the quality and outcomes framework indicator scores predicted mortality in any year.“ [3]
Roland rightly notes that the biggest changes in performance occurred in the first two years of the QOF, which we could not assess. We raised this point in the discussion section of the paper. That said, whilst average performance reached a plateau for most indicators by year 3, individual practices continued to improve and wide variations in performance persisted. If the QOF did have an impact on mortality in its first two years, this was presumably mediated through practice improvement on the clinical indicators. If this mechanism operated in the first two years it should still have been operating in later years and been detectable in our analyses, which investigated differences in performance within practices across years, and between practices within years. Variability in our performance measures was similar to variability for the aggregate comorbidity measures we included in the models, and the latter were, as expected, associated with mortality. Furthermore, in a recent practice-level study Dusheiko et al investigated the relationship between mortality and performance on QOF indicators over the first four years of the scheme, including the early period when the greatest overall improvements occurred [4]. Of the ten conditions they investigated, only quality of care for stroke was significantly associated with lower overall mortality rates.
With respect to our previous research cited by Roland, the 2014 BMJ paper [5] was not published at the original time of writing, and so we cited previous work, concentrating on studies that had specifically investigated the relationship between practice performance and hospital admissions (rather than overall changes in admissions before and after the introduction of the QOF). It is worth noting that the overall reductions in emergency admissions identified in the 2014 paper were driven by large improvements around CHD, and to a lesser extent stroke, and that substantial impacts on admissions were not apparent for most of the QOF conditions assessed. The 2008 Lancet paper (by two of the authors) specifically investigated the relationship between levels of achievement by practices and area deprivation in their locality. [6] Again, whilst the 2008 study found that average levels of achievement plateaued by Year 3 of the scheme, there were still wide variations in achievement and the worst performing practices remained concentrated in the most deprived areas. More pertinently, the 2008 paper did not investigate the impact of either performance or deprivation on mortality, which is the focus of the current paper.
On the points raised by Lewis, we argue that: 1) although we did investigate the overall aggregate of performance, our key message was that mortality was not associated with the aggregate of nine key intermediate outcome indicators that would be expected to improve outcomes, rather the effect of deprivation overshadowed all other relationships; 2) almost all practices participate in the QOF, and no comparator group is available – as noted above, there is substantial variation between practices in achievement for intermediate outcomes indicators; 3) the precise purpose of the paper was to estimate practice effects at a meaningfully small geographical level. The advantages and disadvantages of this analytical approach, compared to a practice level analysis, are discussed in the paper. Nevertheless, as noted above, recent practice-level analyses have produced similar results to our study; 4) visual inspection of graphs can be misleading, and the apparent relationship between morbidity (red) and performance (blue) was not one we investigated. We focused on performance (blue) and standardised mortality rates (not reported in the graphs) and in our regression analyses we found no relationship between them.
In summary, we appreciate that the relationships we investigated are complex and acknowledge that the research community may never manage fully to quantify the direct and indirect effects of the QOF, so the community is unlikely to agree on these effects. However, what we have found at the small area level is that higher overall performance on QOF indicators is not associated with lower mortality rates for key incentivised conditions. Dusheiko et al have produced similar results at the practice level. We need to understand why this is. Patient-level analyses investigating specific diseases, in isolation and in combination, may help to solve the conundrum.
References:
1. Honeyford K, Baker R, Bankart MJG, et al. Estimating smoking prevalence in general practice using data from the Quality and Outcomes Framework (QOF). BMJ Open 2014;4: e005217. doi:10.1136/ bmjopen-2014-005217.
2. Honeyford K, Baker R, Bankart MJG, Jones DR. Modelling factors in primary care quality improvement: a cross-sectional study of premature CHD mortality. BMJ Open 2013;3:e003391 doi:10.1136/bmjopen-2013-003391.
3. Levene LS, Baker R, Bankart MJG, Khunti K. Association of features of primary health care with coronary heart disease mortality. JAMA 2010;304(18):2028–34.
4. Dusheiko M, Gravelle H, Martin S, Smith PC. Quality of Disease Management and Risk of Mortality in English Primary Care Practices. Health Serv Res 2015; DOI: 10.1111/1475-6773.12283
5. Harrison MJ, Dusheiko M, Sutton M, Gravelle H, Doran T, Roland M. Effect of a national primary care pay for performance scheme on emergency hospital admissions for ambulatory care sensitive conditions: controlled longitudinal study. BMJ 2014;349:g6423–3.
6. Doran T, Fullwood C, Kontopantelis E, Reeves D. Effect of financial incentives on inequalities in the delivery of primary clinical care in England: analysis of clinical activity indicators for the quality and outcomes framework. Lancet 2008 Aug 30;372(9640):728–36.
Competing interests:
No competing interests
23 March 2015
Evangelos Kontopantelis
Researcher
Springate DA, Ashworth M, Webb RT, Buchan IE and Doran T
Rapid Response:
Re: Investigating the relationship between quality of primary care and premature mortality in England: a spatial whole-population study
We appreciate the comments of all the correspondents who took time to read and respond to our paper. Before addressing the tangible points raised, we want to make it clear what we did not say in the paper: that primary care does not save lives; that the investment made in the QOF was squandered; that the effort made by thousands of practice staff in meeting QOF targets was wasted; nor that the QOF has had no impact on patient care and outcomes. Furthermore, we cited evidence on the value of primary care in reducing premature mortality, and we have previously contributed evidence on positive impacts of the QOF.
Improving the management of common chronic conditions has been a worthwhile achievement by practices, but it does not necessarily translate into longer lives for patients. In our latest paper, we therefore asked whether there was any association between practices’ performance on the QOF and mortality rates in the populations they serve. Some of the respondents may have preferred us to ask slightly different questions.
We agree with Honeyford et al that more in-depth investigation of specific pathways of care is warranted, however our aim was to investigate whether there is any association between overall QOF performance and mortality. This is an important policy question. Our third performance metric did address intermediate outcomes related to deaths from specific conditions, but even using this measure we could find no association with mortality. On the correspondents’ other points: we did not use recorded prevalence as a measure of primary care quality because this is not directly incentivised under the QOF, and we took the alternative view that register sizes depend on ‘true’ prevalence in the community (which in turn is partly dependent on survival), not just on completeness of case finding. We would also argue that a better measure of practice performance on hypertension screening might be the relevant public health/organisational indicator: the proportion of patients aged over 40/45 with a blood pressure recording. With respect to smoking, although the correspondents have produced a robust method for estimating practice-level smoking rates from QOF indicators [1], it is only applicable from 2012/13, when the relevant indicators were introduced. However, the mortality risk from tobacco use is mediated primarily through cancer, cardiovascular and respiratory disease: we included denominators for these conditions in our models, in addition to area deprivation, which is also strongly correlated with smoking rates.
We were not too surprised that our findings at the LSOA level were not consistent with those of Honeyford et al at the PCT level. PCT-level analyses aggregate heterogeneous areas with respect to the strongest predictors of mortality, such as area deprivation, and we took a different approach to key covariates: for example, our models included rurality and continuous (rather than binary) measures of age. However, we note that only one of the QOF quality indicators modelled in Honeyford et al’s 2013 study reached borderline significance [2], and their 2010 study concluded that “neither provision of primary health care… nor clinical performance as reflected by the quality and outcomes framework indicator scores predicted mortality in any year.“ [3]
Roland rightly notes that the biggest changes in performance occurred in the first two years of the QOF, which we could not assess. We raised this point in the discussion section of the paper. That said, whilst average performance reached a plateau for most indicators by year 3, individual practices continued to improve and wide variations in performance persisted. If the QOF did have an impact on mortality in its first two years, this was presumably mediated through practice improvement on the clinical indicators. If this mechanism operated in the first two years it should still have been operating in later years and been detectable in our analyses, which investigated differences in performance within practices across years, and between practices within years. Variability in our performance measures was similar to variability for the aggregate comorbidity measures we included in the models, and the latter were, as expected, associated with mortality. Furthermore, in a recent practice-level study Dusheiko et al investigated the relationship between mortality and performance on QOF indicators over the first four years of the scheme, including the early period when the greatest overall improvements occurred [4]. Of the ten conditions they investigated, only quality of care for stroke was significantly associated with lower overall mortality rates.
With respect to our previous research cited by Roland, the 2014 BMJ paper [5] was not published at the original time of writing, and so we cited previous work, concentrating on studies that had specifically investigated the relationship between practice performance and hospital admissions (rather than overall changes in admissions before and after the introduction of the QOF). It is worth noting that the overall reductions in emergency admissions identified in the 2014 paper were driven by large improvements around CHD, and to a lesser extent stroke, and that substantial impacts on admissions were not apparent for most of the QOF conditions assessed. The 2008 Lancet paper (by two of the authors) specifically investigated the relationship between levels of achievement by practices and area deprivation in their locality. [6] Again, whilst the 2008 study found that average levels of achievement plateaued by Year 3 of the scheme, there were still wide variations in achievement and the worst performing practices remained concentrated in the most deprived areas. More pertinently, the 2008 paper did not investigate the impact of either performance or deprivation on mortality, which is the focus of the current paper.
On the points raised by Lewis, we argue that: 1) although we did investigate the overall aggregate of performance, our key message was that mortality was not associated with the aggregate of nine key intermediate outcome indicators that would be expected to improve outcomes, rather the effect of deprivation overshadowed all other relationships; 2) almost all practices participate in the QOF, and no comparator group is available – as noted above, there is substantial variation between practices in achievement for intermediate outcomes indicators; 3) the precise purpose of the paper was to estimate practice effects at a meaningfully small geographical level. The advantages and disadvantages of this analytical approach, compared to a practice level analysis, are discussed in the paper. Nevertheless, as noted above, recent practice-level analyses have produced similar results to our study; 4) visual inspection of graphs can be misleading, and the apparent relationship between morbidity (red) and performance (blue) was not one we investigated. We focused on performance (blue) and standardised mortality rates (not reported in the graphs) and in our regression analyses we found no relationship between them.
In summary, we appreciate that the relationships we investigated are complex and acknowledge that the research community may never manage fully to quantify the direct and indirect effects of the QOF, so the community is unlikely to agree on these effects. However, what we have found at the small area level is that higher overall performance on QOF indicators is not associated with lower mortality rates for key incentivised conditions. Dusheiko et al have produced similar results at the practice level. We need to understand why this is. Patient-level analyses investigating specific diseases, in isolation and in combination, may help to solve the conundrum.
References:
1. Honeyford K, Baker R, Bankart MJG, et al. Estimating smoking prevalence in general practice using data from the Quality and Outcomes Framework (QOF). BMJ Open 2014;4: e005217. doi:10.1136/ bmjopen-2014-005217.
2. Honeyford K, Baker R, Bankart MJG, Jones DR. Modelling factors in primary care quality improvement: a cross-sectional study of premature CHD mortality. BMJ Open 2013;3:e003391 doi:10.1136/bmjopen-2013-003391.
3. Levene LS, Baker R, Bankart MJG, Khunti K. Association of features of primary health care with coronary heart disease mortality. JAMA 2010;304(18):2028–34.
4. Dusheiko M, Gravelle H, Martin S, Smith PC. Quality of Disease Management and Risk of Mortality in English Primary Care Practices. Health Serv Res 2015; DOI: 10.1111/1475-6773.12283
5. Harrison MJ, Dusheiko M, Sutton M, Gravelle H, Doran T, Roland M. Effect of a national primary care pay for performance scheme on emergency hospital admissions for ambulatory care sensitive conditions: controlled longitudinal study. BMJ 2014;349:g6423–3.
6. Doran T, Fullwood C, Kontopantelis E, Reeves D. Effect of financial incentives on inequalities in the delivery of primary clinical care in England: analysis of clinical activity indicators for the quality and outcomes framework. Lancet 2008 Aug 30;372(9640):728–36.
Competing interests: No competing interests