Re: PACE trial authors’ reply to letter by Kindlon
Sean Lynch reports that, in his view, the protocol changes for the PACE trial appear well justified[1]. However, the accuracy of the factual claims used by the PACE trial researchers to explain their post-hoc outcome measures should be checked.
The PACE trial's published protocol defined 'recovery' as requiring an SF-36 Physical Functioning (SF36-PF) questionnaire score of at least 85 out of 100, while the trial's entry criteria required a score of 65 or under, which was taken to indicate that patients' fatigue was disabling[2]. The post-hoc criteria for recovery allowed patients with an SF36-PF score of 60 to be classed as recovered. This change was justified by the claim that a threshold of 85 would mean “approximately half the general working age population would fall outside the normal range.”[3] In fact, the data cited showed that the median score for the working age population was 100, less than 18% of the general working age population had a score under 85, and 15% had declared a long-term health problem[4,5].
An SF36-PF score of 60 was claimed in the Lancet PACE paper to be the mean -1sd of the working age population, and thus a suitable threshold for ‘normal’ disability[6]. They had in fact used data which included all those aged over 65, reducing the mean physical function score and increasing the SD[4]. For the working age population the mean -1sd was over 70, requiring patients to score at least 75 to fall within this ‘normal range’[5]. Also, the trial's protocol makes it clear that the thresholds for recovery (including ≥85 for SF-36 PF) were intended to be more demanding than those for the mean -1sd, reporting that: “A score of 70 is about one standard deviation below the mean... for the UK adult population”[2].
The post-hoc criteria for recovery so clearly overlapped with the trial's own criteria for severe and disabling fatigue that an additional element needed to be introduced, mandating that ‘recovered’ patients not also fulfil every aspect of the trial's criteria for CFS[3]. Even so, patients could still have been classed as recovered when reporting no change, or even a decline, in either one of the trial’s primary outcomes.
Even using the loose post-hoc criteria for recovery, only 22% of patients were classed as recovered following treatment with specialist medical care and additional CBT or GET[3]. Regardless, the BMJ had reported that PACE showed CBT and GET “cured” 30% and 28% of patients respectively[7], a Lancet commentary claimed that about 30% recovered using a “strict criterion” for recovery[8], and a paper aimed at NHS commissioners stated PACE indicated a recovery rate of 30-40% for CBT and GET[9,10]. It is wrong for such misstatements of fact to be allowed to go on affecting how doctors treat their patients, how funding decisions are made, and the information that patients are provided with before deciding whether to consent to particular interventions.
The changes to the outcome measures used in the PACE trial may not be “representative of a hidden agenda”[1], but they were misguided, justified by inaccurate claims, and have been misleading to others. The refusal to allow patients access to data on the outcome measures laid out in the trial’s protocol reflects a sad dismissal of their right to be informed about the medical treatments they are being encouraged to pursue[11,12,13].
Lynch goes on to recommend the use of pragmatic trials as a way forward for CFS/ME research[1]. This is unlikely to be helpful. In response to the paper cited by Lynch, professor of complementary medicine Edzard Ernst pointed out that: “drawbacks might mean that the cmRCT generates false positive results. I can, for instance, imagine a pure placebo, like homeopathy, coming out of such a test smelling of roses.”[14] There was a time when it was claimed by some that homeopathy was a promising medical treatment. It is now more widely recognised that homeopathy can simply affect how patients report their symptoms in non-blinded trials, and that it is not ethical to promote it as a legitimate form of medicine.
In the case of cognitive and behavioural interventions for CFS/ME, we have evidence from the PACE trial that they are able to lead to modest improvements in patient questionnaire scores in a non-blinded trial, without leading to improvements in real world outcomes such as employment rates, or claims for disability benefits[15]. A meta-analysis of actometer data from CBT trials for CFS also found that CBT was able to lead to improvements in questionnaire scores in non-blinded trials, but not to improvements in the amount of activity that patients were actually able to perform[16]. Sadly, the PACE trial dropped actometers as an outcome measure, although they were purchased and used at baseline[17].
Recent evidence from a large study of NHS CFS/ME specialist services indicated that reported results for CBT and GET are poorer than those reported in PACE, and that centres offering CBT and GET achieved marginally worse results than centres offering ‘activity management’[18]. We do not currently have compelling evidence that CBT or GET are more effective medical interventions for ME/CFS than homeopathy, despite some of the claims made by proponents.
It should be seen as no more acceptable for those with financial, professional or ideological interests in promoting CBT or GET as treatments for ME/CFS to exaggerate the value of these interventions than it is for others to exaggerate the value of homeopathy. Anyone with a real interest in helping patients with ME/CFS, and in allowing them to make informed decisions about their own health care, should now call for the release of results for all of the outcomes laid out in the PACE trial's published protocol[2].
[2] White PD, Sharpe MC, Chalder T, DeCesare JC, Walyin R: Protocol for the PACE trial: a randomised controlled trial of adaptative pacing, cognitive behaviour therapy and graded exercise as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurol 2007, 7:6
[3] White PD, Johnson AL, Goldsmith K, Chalder T, Sharpe MC. Recovery from chronic fatigue syndrome after treatments given in the PACE trial. Psychol Med 2013;1-9, published online 31 Jan. doi:10.1017/S0033291713000020.
[4] Bowling A, Bond M, Jenkinson C, Lamping DL. Short form 36 (SF-36) health survey questionnaire: which normative data should be used? Comparisons between the norms provided by the Omnibus Survey in Britain, The Health Survey for England and the Oxford Healthy Life Survey. J Publ Health Med 1999, 21: 255–70.
[5] Office of Population Censuses and Surveys. Social Survey Division, OPCS Omnibus Survey, November 1992. Colchester, Essex: UK Data Archive, September 1997. SN: 3660, http://dx.doi.org/10.5255/UKDA-SN-3660-1
[6] White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, Baber HL, Burgess M, Clark LV, Cox DL, Bavinton J, Angus BJ, Murphy G, Murphy M, O’Dowd H, Wilks D, McCrone P, Chalder T, Sharpe M. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011;377:823-36.
[7] BMJ Short Cuts: ‘All you need to read in the other general journals’ BMJ 2011;342:d1168
[8] Knoop H, Bleijenberg G. Chronic fatigue syndrome: where to PACE from here?. Lancet 2011; 377: 786-788.
[9] Collin SM, Crawley E, May MT, Sterne JAC, Hollingworth W: The impact of CFS/ME on employment and productivity in the UK: a cross-sectional study based on the CFS/ME national outcomes database. BMC Health Serv Res 2011, 11:217.
[15] McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD. (2012) Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS ONE 7: e40808.
[16] Bleijenberg G, Prins JB, Wiborg JF, Knoop H, Stulemeijer M,. 'How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity.' Psychol Med. 2010 Aug;40(8):1281-7.
[18] Crawley E, Collin SM, White PD, Rimes K, Sterne JA, May MT; CFS/ME National Outcomes Database. (2013) Treatment outcome in adults with chronic fatigue syndrome: a prospective study in England based on the CFS/ME National Outcomes Database. QJM. 106:555-65.
Competing interests:
I believe that the current tolerance for preventing patients from accessing data on the efficacy of the treatments available to them will be looked back upon with a sense of real shame.
Rapid Response:
Re: PACE trial authors’ reply to letter by Kindlon
Sean Lynch reports that, in his view, the protocol changes for the PACE trial appear well justified[1]. However, the accuracy of the factual claims used by the PACE trial researchers to explain their post-hoc outcome measures should be checked.
The PACE trial's published protocol defined 'recovery' as requiring an SF-36 Physical Functioning (SF36-PF) questionnaire score of at least 85 out of 100, while the trial's entry criteria required a score of 65 or under, which was taken to indicate that patients' fatigue was disabling[2]. The post-hoc criteria for recovery allowed patients with an SF36-PF score of 60 to be classed as recovered. This change was justified by the claim that a threshold of 85 would mean “approximately half the general working age population would fall outside the normal range.”[3] In fact, the data cited showed that the median score for the working age population was 100, less than 18% of the general working age population had a score under 85, and 15% had declared a long-term health problem[4,5].
An SF36-PF score of 60 was claimed in the Lancet PACE paper to be the mean -1sd of the working age population, and thus a suitable threshold for ‘normal’ disability[6]. They had in fact used data which included all those aged over 65, reducing the mean physical function score and increasing the SD[4]. For the working age population the mean -1sd was over 70, requiring patients to score at least 75 to fall within this ‘normal range’[5]. Also, the trial's protocol makes it clear that the thresholds for recovery (including ≥85 for SF-36 PF) were intended to be more demanding than those for the mean -1sd, reporting that: “A score of 70 is about one standard deviation below the mean... for the UK adult population”[2].
The post-hoc criteria for recovery so clearly overlapped with the trial's own criteria for severe and disabling fatigue that an additional element needed to be introduced, mandating that ‘recovered’ patients not also fulfil every aspect of the trial's criteria for CFS[3]. Even so, patients could still have been classed as recovered when reporting no change, or even a decline, in either one of the trial’s primary outcomes.
Even using the loose post-hoc criteria for recovery, only 22% of patients were classed as recovered following treatment with specialist medical care and additional CBT or GET[3]. Regardless, the BMJ had reported that PACE showed CBT and GET “cured” 30% and 28% of patients respectively[7], a Lancet commentary claimed that about 30% recovered using a “strict criterion” for recovery[8], and a paper aimed at NHS commissioners stated PACE indicated a recovery rate of 30-40% for CBT and GET[9,10]. It is wrong for such misstatements of fact to be allowed to go on affecting how doctors treat their patients, how funding decisions are made, and the information that patients are provided with before deciding whether to consent to particular interventions.
The changes to the outcome measures used in the PACE trial may not be “representative of a hidden agenda”[1], but they were misguided, justified by inaccurate claims, and have been misleading to others. The refusal to allow patients access to data on the outcome measures laid out in the trial’s protocol reflects a sad dismissal of their right to be informed about the medical treatments they are being encouraged to pursue[11,12,13].
Lynch goes on to recommend the use of pragmatic trials as a way forward for CFS/ME research[1]. This is unlikely to be helpful. In response to the paper cited by Lynch, professor of complementary medicine Edzard Ernst pointed out that: “drawbacks might mean that the cmRCT generates false positive results. I can, for instance, imagine a pure placebo, like homeopathy, coming out of such a test smelling of roses.”[14] There was a time when it was claimed by some that homeopathy was a promising medical treatment. It is now more widely recognised that homeopathy can simply affect how patients report their symptoms in non-blinded trials, and that it is not ethical to promote it as a legitimate form of medicine.
In the case of cognitive and behavioural interventions for CFS/ME, we have evidence from the PACE trial that they are able to lead to modest improvements in patient questionnaire scores in a non-blinded trial, without leading to improvements in real world outcomes such as employment rates, or claims for disability benefits[15]. A meta-analysis of actometer data from CBT trials for CFS also found that CBT was able to lead to improvements in questionnaire scores in non-blinded trials, but not to improvements in the amount of activity that patients were actually able to perform[16]. Sadly, the PACE trial dropped actometers as an outcome measure, although they were purchased and used at baseline[17].
Recent evidence from a large study of NHS CFS/ME specialist services indicated that reported results for CBT and GET are poorer than those reported in PACE, and that centres offering CBT and GET achieved marginally worse results than centres offering ‘activity management’[18]. We do not currently have compelling evidence that CBT or GET are more effective medical interventions for ME/CFS than homeopathy, despite some of the claims made by proponents.
It should be seen as no more acceptable for those with financial, professional or ideological interests in promoting CBT or GET as treatments for ME/CFS to exaggerate the value of these interventions than it is for others to exaggerate the value of homeopathy. Anyone with a real interest in helping patients with ME/CFS, and in allowing them to make informed decisions about their own health care, should now call for the release of results for all of the outcomes laid out in the PACE trial's published protocol[2].
[1] Lynch S. Re: PACE trial authors’ reply to letter by Kindlon. BMJ Rapid Response.http://www.bmj.com/content/347/bmj.f5963/rr/671093
[2] White PD, Sharpe MC, Chalder T, DeCesare JC, Walyin R: Protocol for the PACE trial: a randomised controlled trial of adaptative pacing, cognitive behaviour therapy and graded exercise as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurol 2007, 7:6
[3] White PD, Johnson AL, Goldsmith K, Chalder T, Sharpe MC. Recovery from chronic fatigue syndrome after treatments given in the PACE trial. Psychol Med 2013;1-9, published online 31 Jan. doi:10.1017/S0033291713000020.
[4] Bowling A, Bond M, Jenkinson C, Lamping DL. Short form 36 (SF-36) health survey questionnaire: which normative data should be used? Comparisons between the norms provided by the Omnibus Survey in Britain, The Health Survey for England and the Oxford Healthy Life Survey. J Publ Health Med 1999, 21: 255–70.
[5] Office of Population Censuses and Surveys. Social Survey Division, OPCS Omnibus Survey, November 1992. Colchester, Essex: UK Data Archive, September 1997. SN: 3660, http://dx.doi.org/10.5255/UKDA-SN-3660-1
[6] White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, Baber HL, Burgess M, Clark LV, Cox DL, Bavinton J, Angus BJ, Murphy G, Murphy M, O’Dowd H, Wilks D, McCrone P, Chalder T, Sharpe M. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011;377:823-36.
[7] BMJ Short Cuts: ‘All you need to read in the other general journals’ BMJ 2011;342:d1168
[8] Knoop H, Bleijenberg G. Chronic fatigue syndrome: where to PACE from here?. Lancet 2011; 377: 786-788.
[9] Collin SM, Crawley E, May MT, Sterne JAC, Hollingworth W: The impact of CFS/ME on employment and productivity in the UK: a cross-sectional study based on the CFS/ME national outcomes database. BMC Health Serv Res 2011, 11:217.
[10] Interview with Amy Chesterton and Esther Crawley. Available at http://www.thenakedscientists.com/HTML/content/news-archive/news/2384/
[11] Freedom of Information request http://www.meassociation.org.uk/?p=6171
[12] Freedom of Information response http://www.meassociation.org.uk/wp-content/uploads/2011/06/FOI from Queen Mary.pdf [3]
[13] Follow up Freedom of Information request: https://www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po
[14] Ernst, E. The cmRCT BMJ Rapid Response. http://www.bmj.com/rapid-response/2011/11/02/cmrct
[15] McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD. (2012) Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS ONE 7: e40808.
[16] Bleijenberg G, Prins JB, Wiborg JF, Knoop H, Stulemeijer M,. 'How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity.' Psychol Med. 2010 Aug;40(8):1281-7.
[17] PD White, MC Sharpe, T Chalder, JC DeCesare, R Walwyn, for the PACE trial management group: Response to comments on "Protocol for the PACE trial" http://www.biomedcentral.com/1471-2377/7/6/comments#306608
[18] Crawley E, Collin SM, White PD, Rimes K, Sterne JA, May MT; CFS/ME National Outcomes Database. (2013) Treatment outcome in adults with chronic fatigue syndrome: a prospective study in England based on the CFS/ME National Outcomes Database. QJM. 106:555-65.
Competing interests: I believe that the current tolerance for preventing patients from accessing data on the efficacy of the treatments available to them will be looked back upon with a sense of real shame.