In the joint stakeholder position statement referred to in the above article, six methodological issues were raised concerning the 2018 NICE draft depression guideline. These were: that patient experience research has been left out; that long-term outcome data has been left out; that chronic and complex forms of depression have been separated into misleading categories of depression; that severity has been misrepresented, disadvantaging people with the most severe forms of depression; that an experimental and confounded network meta-analysis has been used as a primary rather than supplementary basis for recommendations; that quality of life outcomes have been left out(1).
In a series of articles written with experts in the field, I have looked at some of these methodological issues in detail and provide an overview of this series here.
1. Why should NICE include patient experience research?
Prof Peter Beresford and I argue that NICE have conflated issues of patient involvement in the depression guideline development with patient choice and patient experience research (2). By choosing not to conduct a systematic review of patient experience research, a significant body of literature is being excluded. In a metasynthesis currently being undertaken with colleagues on patient experiences of psychological treatments for depression, I have identified 38 studies for inclusion, only one of which is mentioned in the draft NICE guideline. These studies should be reviewed by NICE along with a further review examining patient experiences of antidepressants. The guideline should also take into account experiences of carers. In a recent metasynthesis of carers’ experiences of depression, Jemma Priestley and I reviewed 15 studies, none of which have been taken into account in the guideline (3).
2. Why should NICE examine long-term outcomes?
Michael P. Hengartner and I have undertaken an analysis of long-term outcomes for the trials of treatments for long-term depression. We illustrate how and why NICE do so too (4). Although long-term evidence from RCTs is limited in volume, the findings from this set of data (that psychological therapies are superior to antidepressants in the long-term) is supported by a range of systematic reviews of other forms of evidence including naturalistic cohort studies and discontinuation trials. NICE have produced recommendations for depression treatment for 15 years now, ignoring this data and potentially contributing to the crisis of prescribed drug dependency now recognised by Public Health England (https://phe-newsroom.prgloo.com/news/dependence-on-prescription-medicine...).
3. What’s wrong with subgrouping chronic forms of depression and misrepresenting severity?
NICE have made fundamental philosophical errors in separating ‘treatment resistant depression’, ‘chronic’ and ‘complex’ depression(5). I argue that in making these distinctions without having full knowledge of individuals’ experiences, the argument that these sub-classifications of persistent forms of depression are clinically useful dissolves and becomes more like a game of Minecraft in which cuboids are built up into logically impossible structures. This applies to the separation of ‘mild’ and ‘severe’ depression which is based on a flawed approach and creates a false sense of certainty. There is therefore no good justification for artificially separating trials into these categories in the guideline and doing so leads to misleading recommendations.
4. Why is quality of life important?
NICE claim that it is not possible to look at quality of life outcomes because use of these measures is rare:
“We agree that measures of functioning and quality of life measures are important. However, these measures are relatively rarely reported. For this reason these measures were not prioritised for inclusion in the review protocols for this guideline.”(6)
An examination of the appendices to the draft guideline reveals that of the 124 trials reviewed in the treatment resistant and chronic depression categories alone, 58 (nearly half) use a quality of life or functioning measure. It is true that the specific measures used vary a great deal; there are 16 different measures among these trials. However, there were 13 different symptom outcome measures and NICE took all of these into account.
It is important to examine quality of life outcomes not least because NICE have consistently stated in their own scoping documents and previous guidelines that quality of life is an important outcome; quality of life is also a central pillar of government policy (e.g. No Health Without Mental Health). Moreover, service users often note that outcomes such as better relationships and better functioning matter more than symptom improvement. Although some of the measures typically used may not perfectly reflect service user priorities(7), it would indicate a move in the right direction to demonstrate commitment to examining patient preferred outcomes. NICE may consider that the results would not be any different; yet in a re-analysis of the first NICE depression guideline (2004), colleagues and I demonstrated that analysing quality of life outcome measures does produce different findings in reviews(8).
Some may take the view that ‘interfering’ in NICE guideline development is antithetical to the nature of our UK guideline body. Indeed NICE was established to be free from political interference and lobbying from pharmaceutical companies. However, Professor Maurice Sunkin QC and I make the case that the stakeholder consultation process is a critical element of quality assurance for NICE guidelines (9). Stakeholder consultation can only serve as a robust quality assurance process if legitimate methodological concerns raised by stakeholders during this process are taken into account and not brushed off as minor inconveniences. Moreover, as a public body, NICE is bound by English law to consult in good faith and not to have made decisions prior to the consultation.
Taken together, it seems likely that unless the methodological concerns outlined are addressed in full in the next draft of the guideline, stakeholders may well continue to reject the guideline as unfit-for-purpose.
1. Rost, F. & McPherson, S. (2018). Joint stakeholder position statement on the NICE guideline for depression in adults. 2018. https://www.nsun.org.uk/news/stakeholder-position-statement-on-the-nice-...
2. McPherson, S., & Beresford, P. (2019). Semantics of patient choice: how the UK national guideline for depression silences patients. Disability and Society, 1-7. doi:10.1080/09687599.2019.1589757
3. Priestley, J., & McPherson, S. (2016). Experiences of adults providing care to a partner or relative with depression: A meta-ethnographic synthesis. Journal of Affective Disorders, 192, 41-49. doi:10.1016/j.jad.2015.12.011
4. McPherson, S. & Hengartner, M.P. (2019). Long-term outcomes of trials in the NICE depression guideline. BJPsych Open, 5(5). https://doi.org/10.1192/bjo.2019.65
5. McPherson, S. (2019). A NICE game of Minecraft: philosophical flaws underpinning UK depression guideline nosology. Medical Humanities. doi:10.1136/medhum-2019-011658
6. NICE. (2018). Consultation comments and responses – second consultation period. https://www.nice.org.uk/guidance/gid-cgwave0725/documents/consultation-c...
7. Connell, J., O'Cathain, A., Brazier, J. (2014). Measuring quality of life in mental health: Are we asking the right questions? Social Science & Medicine, 120, 12-20. 10.1016/j.socscimed.2014.08.026
8. McPherson, S., Evans, C., & Richardson, P. (2009). The NICE Depression Guidelines and the recovery model: Is there an evidence base for IAPT?. Journal of Mental Health, 18(5), 405-414. Retrieved from http://dx.doi.org/10.3109/09638230902968258
9. McPherson, S., & Sunkin, M. (2019). The Dobson–Rawlins pact and the National Institute for Health and Care Excellence: impact of political independence on scientific and legal accountability. The British Journal of Psychiatry. doi:10.1192/bjp.2019.121
Rapid Response:
What's wrong with the NICE depression guideline?
In the joint stakeholder position statement referred to in the above article, six methodological issues were raised concerning the 2018 NICE draft depression guideline. These were: that patient experience research has been left out; that long-term outcome data has been left out; that chronic and complex forms of depression have been separated into misleading categories of depression; that severity has been misrepresented, disadvantaging people with the most severe forms of depression; that an experimental and confounded network meta-analysis has been used as a primary rather than supplementary basis for recommendations; that quality of life outcomes have been left out(1).
In a series of articles written with experts in the field, I have looked at some of these methodological issues in detail and provide an overview of this series here.
1. Why should NICE include patient experience research?
Prof Peter Beresford and I argue that NICE have conflated issues of patient involvement in the depression guideline development with patient choice and patient experience research (2). By choosing not to conduct a systematic review of patient experience research, a significant body of literature is being excluded. In a metasynthesis currently being undertaken with colleagues on patient experiences of psychological treatments for depression, I have identified 38 studies for inclusion, only one of which is mentioned in the draft NICE guideline. These studies should be reviewed by NICE along with a further review examining patient experiences of antidepressants. The guideline should also take into account experiences of carers. In a recent metasynthesis of carers’ experiences of depression, Jemma Priestley and I reviewed 15 studies, none of which have been taken into account in the guideline (3).
2. Why should NICE examine long-term outcomes?
Michael P. Hengartner and I have undertaken an analysis of long-term outcomes for the trials of treatments for long-term depression. We illustrate how and why NICE do so too (4). Although long-term evidence from RCTs is limited in volume, the findings from this set of data (that psychological therapies are superior to antidepressants in the long-term) is supported by a range of systematic reviews of other forms of evidence including naturalistic cohort studies and discontinuation trials. NICE have produced recommendations for depression treatment for 15 years now, ignoring this data and potentially contributing to the crisis of prescribed drug dependency now recognised by Public Health England (https://phe-newsroom.prgloo.com/news/dependence-on-prescription-medicine...).
3. What’s wrong with subgrouping chronic forms of depression and misrepresenting severity?
NICE have made fundamental philosophical errors in separating ‘treatment resistant depression’, ‘chronic’ and ‘complex’ depression(5). I argue that in making these distinctions without having full knowledge of individuals’ experiences, the argument that these sub-classifications of persistent forms of depression are clinically useful dissolves and becomes more like a game of Minecraft in which cuboids are built up into logically impossible structures. This applies to the separation of ‘mild’ and ‘severe’ depression which is based on a flawed approach and creates a false sense of certainty. There is therefore no good justification for artificially separating trials into these categories in the guideline and doing so leads to misleading recommendations.
4. Why is quality of life important?
NICE claim that it is not possible to look at quality of life outcomes because use of these measures is rare:
“We agree that measures of functioning and quality of life measures are important. However, these measures are relatively rarely reported. For this reason these measures were not prioritised for inclusion in the review protocols for this guideline.”(6)
An examination of the appendices to the draft guideline reveals that of the 124 trials reviewed in the treatment resistant and chronic depression categories alone, 58 (nearly half) use a quality of life or functioning measure. It is true that the specific measures used vary a great deal; there are 16 different measures among these trials. However, there were 13 different symptom outcome measures and NICE took all of these into account.
It is important to examine quality of life outcomes not least because NICE have consistently stated in their own scoping documents and previous guidelines that quality of life is an important outcome; quality of life is also a central pillar of government policy (e.g. No Health Without Mental Health). Moreover, service users often note that outcomes such as better relationships and better functioning matter more than symptom improvement. Although some of the measures typically used may not perfectly reflect service user priorities(7), it would indicate a move in the right direction to demonstrate commitment to examining patient preferred outcomes. NICE may consider that the results would not be any different; yet in a re-analysis of the first NICE depression guideline (2004), colleagues and I demonstrated that analysing quality of life outcome measures does produce different findings in reviews(8).
Some may take the view that ‘interfering’ in NICE guideline development is antithetical to the nature of our UK guideline body. Indeed NICE was established to be free from political interference and lobbying from pharmaceutical companies. However, Professor Maurice Sunkin QC and I make the case that the stakeholder consultation process is a critical element of quality assurance for NICE guidelines (9). Stakeholder consultation can only serve as a robust quality assurance process if legitimate methodological concerns raised by stakeholders during this process are taken into account and not brushed off as minor inconveniences. Moreover, as a public body, NICE is bound by English law to consult in good faith and not to have made decisions prior to the consultation.
Taken together, it seems likely that unless the methodological concerns outlined are addressed in full in the next draft of the guideline, stakeholders may well continue to reject the guideline as unfit-for-purpose.
1. Rost, F. & McPherson, S. (2018). Joint stakeholder position statement on the NICE guideline for depression in adults. 2018. https://www.nsun.org.uk/news/stakeholder-position-statement-on-the-nice-...
2. McPherson, S., & Beresford, P. (2019). Semantics of patient choice: how the UK national guideline for depression silences patients. Disability and Society, 1-7. doi:10.1080/09687599.2019.1589757
3. Priestley, J., & McPherson, S. (2016). Experiences of adults providing care to a partner or relative with depression: A meta-ethnographic synthesis. Journal of Affective Disorders, 192, 41-49. doi:10.1016/j.jad.2015.12.011
4. McPherson, S. & Hengartner, M.P. (2019). Long-term outcomes of trials in the NICE depression guideline. BJPsych Open, 5(5). https://doi.org/10.1192/bjo.2019.65
5. McPherson, S. (2019). A NICE game of Minecraft: philosophical flaws underpinning UK depression guideline nosology. Medical Humanities. doi:10.1136/medhum-2019-011658
6. NICE. (2018). Consultation comments and responses – second consultation period. https://www.nice.org.uk/guidance/gid-cgwave0725/documents/consultation-c...
7. Connell, J., O'Cathain, A., Brazier, J. (2014). Measuring quality of life in mental health: Are we asking the right questions? Social Science & Medicine, 120, 12-20. 10.1016/j.socscimed.2014.08.026
8. McPherson, S., Evans, C., & Richardson, P. (2009). The NICE Depression Guidelines and the recovery model: Is there an evidence base for IAPT?. Journal of Mental Health, 18(5), 405-414. Retrieved from http://dx.doi.org/10.3109/09638230902968258
9. McPherson, S., & Sunkin, M. (2019). The Dobson–Rawlins pact and the National Institute for Health and Care Excellence: impact of political independence on scientific and legal accountability. The British Journal of Psychiatry. doi:10.1192/bjp.2019.121
Competing interests: No competing interests