Re: PACE trial authors’ reply to letter by Kindlon
Contrary to Prof. Lynch's view, there remain critical issues with how data from the PACE trial has been generated, interpreted, and presented, the general nature of which he is aware [1,2].
These issues include: using only subjective self-report measures for primary outcome; failing to report results fully and appropriately; inadequate use of objective measures; and discrepancies between subjective and objective measures, a known problem in this area of medicine [3].
PACE reports the results on both primary measures at outcome (one year follow-up), for the Cognitive Behavioural Therapy (CBT) and Graded Exercise Therapy (GET) arms, were that "No more than 30% of participants were within normal ranges..." [4].
However, all four trial arms received Specialist Medical Care (SMC) which alone accounts for a 15% improvement, so no more than an additional 15% of patients reported a "normal" range outcome following those two treatments, with questions about how that normal was more loosely re-defined at the analysis stage of this non-blinded study.
There was no difference between the CBT, Adaptive Pacing Therapy (APT), and SMC arms on the only objective physical measure reported at outcome (6-Minute Walking Distance Test, 6MWT), and while there was a difference for the GET arm it failed to reach clinical significance (achieving a gain of just 35.3 metres, as mean difference from SMC) [5].
Patients in active treatment arms had a mean age of 40 years (at outcome), but their mean 6MWT scores were worse than healthy 70-80 year olds [6]. They were also worse than Class III heart failure patients, with the only category of cardiopulmonary disease doing worse than PACE being End Stage Lung Disease [7,8].
The 6MWT was not repeated 24-48 hours later, thus failing to objectively quantify post-exertional malaise (PEM), with only a single subjective self-report measure used for this arguably cardinal symptom [9,10].
The uninterrupted walking distance used for the 6MWT was an unusually low 10 metres, a possible confounder [11]. However, more robust data shows no significant difference between shorter and longer uninterrupted distances, and even correcting for the maximum difference found by Ng (17%) the PACE results still do not reach clinical significance [5,11,12].
Outcome data for the 6MWT is only available for 69-74% of patients, compared to the range for self-report measures of 89-97%, with no explanation provided for that missing data, and no data reported on whether the non-responders were different in any way to those who undertook the test. Furthermore, the lowest reporting rate for the 6MWT was in the GET arm, which was also the arm reporting the best 6MWT result. These are potential sources of bias in favour of a reported improvement, because the failure of those patients to take this physical test at outcome may have been due to their deterioration over the course of the trial.
Baseline data was gathered for the two remaining objective physical measures listed in the trial protocol (Self-Paced Step Test - ST, and actigraphy), but no outcome data was reported for either, with no explanation provided for the missing ST data, and a problematic one offered for the omitted actigraphy data [13,14].
The Client Service Receipt Inventory (CSRI) showed no improvement in the more objective "real world" measures of employment participation, reliance on welfare or on disability or income insurance, nor total service usage and costs [15]. Improvement was only shown on some subjective self-report based measures such as the EuroQol (EQ-5D, not yet confirmed as an appropriate instrument for CFS), Clinical Global Impression, HADS anxiety and depression scales, Work and Social Adjustment Scale, symptom count, and PEM [4,15,16,17].
Previous studies have not found that CBT or GET lead to improvements in objective actigraphy results [3]. Nor have they found any correlation (at outcome) between subjective self-report measures and objective actigraphy results [3]: No comparable correlation calculations have been published by PACE for either the 6MWT, or the objective measures in the CSRI. But the lack of statistically significant results for those components of the CSRI, and for the 6MWT in the CBT, APT, and SMC arms, rule out any possible correlations there. As previously noted, the 6MWT results for the GET arm did reach statistical significance but failed to reach clinical significance, so even if a correlation exists here it is unlikely to be of much practical value.
The general pattern of improvement on the two primary outcome measures was that the bulk of it occurred during the initial 3 months of the 6 month long therapy period, then tapered off asymptotically, with decreasing additional improvement over the remainder of the 12 month trial. This pattern of subjective self-report response, combined with the lack of objective correlations, could be explained by confounding generic influences such as priming, and social desirability.
Even with the lax post-hoc criteria for improvement the Number Needed to Treat for both CBT and GET to achieve these results was an unimpressive 1 in 7, with those two treatments only mediating approximately 20% of the already modest total effect [16,18].
PACE also excluded the sickest patients, (those unable to attend a study centre, potentially 25% of the patient population [19]).
Longer term follow-up data has not yet been reported, so we do not know if even these methodologically problematic and clinically marginal results are sustained.
Lastly, probably the two most comparable contemporary trials to PACE (Núñez, and FINE - "the sister study to PACE") failed to support its therapeutic claims [20,21,22].
Collectively these findings do not suggest that the CBT-GET based model employed by PACE has demonstrated sufficient explanatory or therapeutic power upon which to safely and productively build clinical and research programs. If anything they have confirmed the paucity of that model, and the increasingly urgent need to look elsewhere for answers to this troubled, refractory, and distressing condition.
Prof. Stephen Holgate, chair of the UK CFS/ME Research Collaborative, gave a keynote speech recently in which he described our current understanding of this condition as "bathing... ...in ignorance" [23]. I can only agree.
3. Wiborg JF, et al. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity.
Psychol Med. 2010 Aug;40(8):1281-7. doi: 10.1017/S0033291709992212.
4. White PD, Goldsmith KA, Johnson AL, et al, on behalf of the PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial.
Lancet, 2011; published online Feb 18. DOI:10.1016/S0140-6736(11)60096-2.
5. Wise RA, Brown CD. Minimal clinically important differences in the six-minute walk test and the incremental shuttle walking test.
COPD, 2005; 2(1): 1259.
6. Enright PL, Sherrill DL. Reference equations for the six-minute walk in healthy adults.
Am. J. Respir. Crit. Care Med., 1998; 158: 13841387
7. Lipkin DP, Scriven AJ, Crake T, Poole-Wilson PA. (1986) Six minute walking test for assessing exercise capacity in chronic heart failure.
Br Med J (Clin Res Ed). 292:653-5.
8. Ross RM, Murthy JN, Wollak ID, Jackson AS. The six minute walk test accurately estimates mean peak oxygen uptake.
BMC Pulmonary Medicine, 2010, 10: 3
9. VanNess JM, et al. Postexertional malaise in women with chronic fatigue syndrome. J Womens Health (Larchmt). 2010 Feb; 19(2): 239-44.
10. Light AR, et al. Moderate exercise increases expression for sensory, adrenergic, and immune genes in chronic fatigue syndrome patients but not in normal subjects.
J Pain 2009;10:1099
11. Ng SS, et al. Walkway Length, But Not Turning Direction, Determines the Six-Minute Walk Test Distance in Individuals With Stroke.
Archives of Physical Medicine and Rehabilitation: Volume 92, Issue 5 , Pages 806-811, May 2011
12. Sciurba F, et al. Six-minute walk distance in chronic obstructive pulmonary disease: reproducibility and effect of walking course layout and length.
Am J Respir Crit Care Med. 2003 Jun 1;167(11):1522-7. Epub 2003 Feb 20.
13. White PD, Sharpe MC,Chalder T, DeCesare JC and Walwyn R for the PACE trial group. Protocol for the PACE trial: A randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy.
BMC Neurology 2007, 7:6. doi:10.1186/1471-2377-7-6
15. McCrone P, Sharpe M, Chalder T, et al. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis.
PLoS One. 2012;7(8):e40808.
16. P. D. White, K. Goldsmith, A. L. Johnson, T. Chalder and M. Sharpe (2013). Recovery from chronic fatigue syndrome after treatments given in the PACE trial.
Psychological Medicine, 43, pp 2227-2235 doi:10.1017/S0033291713000020
17. Cella M, Sharpe M, Chalder T. Measuring disability in chronic fatigue syndrome: reliability and validity of the Work and Social Adjustment Scale.
J Psychosom Res. 2011 Sep;71(3):124-8. doi: 10.1016/j.jpsychores.2011.02.009.
18. Goldsmith K, et al. How do treatments for chronic fatigue syndrome work? Exploration of instrumental variable methods for mediation analysis in PACE – a randomised controlled trial of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care.
Trials 2011, 12(Suppl 1):A144 doi:10.1186/1745-6215-12-S1-A144
20. Núñez M, et al. Health-related quality of life in patients with chronic fatigue syndrome: group cognitive behavioural therapy and graded exercise versus usual treatment. A randomised controlled trial with 1 year of follow-up
Clin Rheumatol. 2011 Mar;30(3):381-9. doi 10.1007/s10067-010-1677-y
21. Wearden AJ, et al. Nurse led, home based self help treatment for patients in primary care with chronic fatigue syndrome: randomised controlled trial.
BMJ 2010;340:c1777. doi: 10.1136/bmj.c1777.
Rapid Response:
Re: PACE trial authors’ reply to letter by Kindlon
Contrary to Prof. Lynch's view, there remain critical issues with how data from the PACE trial has been generated, interpreted, and presented, the general nature of which he is aware [1,2].
These issues include: using only subjective self-report measures for primary outcome; failing to report results fully and appropriately; inadequate use of objective measures; and discrepancies between subjective and objective measures, a known problem in this area of medicine [3].
PACE reports the results on both primary measures at outcome (one year follow-up), for the Cognitive Behavioural Therapy (CBT) and Graded Exercise Therapy (GET) arms, were that "No more than 30% of participants were within normal ranges..." [4].
However, all four trial arms received Specialist Medical Care (SMC) which alone accounts for a 15% improvement, so no more than an additional 15% of patients reported a "normal" range outcome following those two treatments, with questions about how that normal was more loosely re-defined at the analysis stage of this non-blinded study.
There was no difference between the CBT, Adaptive Pacing Therapy (APT), and SMC arms on the only objective physical measure reported at outcome (6-Minute Walking Distance Test, 6MWT), and while there was a difference for the GET arm it failed to reach clinical significance (achieving a gain of just 35.3 metres, as mean difference from SMC) [5].
Patients in active treatment arms had a mean age of 40 years (at outcome), but their mean 6MWT scores were worse than healthy 70-80 year olds [6]. They were also worse than Class III heart failure patients, with the only category of cardiopulmonary disease doing worse than PACE being End Stage Lung Disease [7,8].
The 6MWT was not repeated 24-48 hours later, thus failing to objectively quantify post-exertional malaise (PEM), with only a single subjective self-report measure used for this arguably cardinal symptom [9,10].
The uninterrupted walking distance used for the 6MWT was an unusually low 10 metres, a possible confounder [11]. However, more robust data shows no significant difference between shorter and longer uninterrupted distances, and even correcting for the maximum difference found by Ng (17%) the PACE results still do not reach clinical significance [5,11,12].
Outcome data for the 6MWT is only available for 69-74% of patients, compared to the range for self-report measures of 89-97%, with no explanation provided for that missing data, and no data reported on whether the non-responders were different in any way to those who undertook the test. Furthermore, the lowest reporting rate for the 6MWT was in the GET arm, which was also the arm reporting the best 6MWT result. These are potential sources of bias in favour of a reported improvement, because the failure of those patients to take this physical test at outcome may have been due to their deterioration over the course of the trial.
Baseline data was gathered for the two remaining objective physical measures listed in the trial protocol (Self-Paced Step Test - ST, and actigraphy), but no outcome data was reported for either, with no explanation provided for the missing ST data, and a problematic one offered for the omitted actigraphy data [13,14].
The Client Service Receipt Inventory (CSRI) showed no improvement in the more objective "real world" measures of employment participation, reliance on welfare or on disability or income insurance, nor total service usage and costs [15]. Improvement was only shown on some subjective self-report based measures such as the EuroQol (EQ-5D, not yet confirmed as an appropriate instrument for CFS), Clinical Global Impression, HADS anxiety and depression scales, Work and Social Adjustment Scale, symptom count, and PEM [4,15,16,17].
Previous studies have not found that CBT or GET lead to improvements in objective actigraphy results [3]. Nor have they found any correlation (at outcome) between subjective self-report measures and objective actigraphy results [3]: No comparable correlation calculations have been published by PACE for either the 6MWT, or the objective measures in the CSRI. But the lack of statistically significant results for those components of the CSRI, and for the 6MWT in the CBT, APT, and SMC arms, rule out any possible correlations there. As previously noted, the 6MWT results for the GET arm did reach statistical significance but failed to reach clinical significance, so even if a correlation exists here it is unlikely to be of much practical value.
The general pattern of improvement on the two primary outcome measures was that the bulk of it occurred during the initial 3 months of the 6 month long therapy period, then tapered off asymptotically, with decreasing additional improvement over the remainder of the 12 month trial. This pattern of subjective self-report response, combined with the lack of objective correlations, could be explained by confounding generic influences such as priming, and social desirability.
Even with the lax post-hoc criteria for improvement the Number Needed to Treat for both CBT and GET to achieve these results was an unimpressive 1 in 7, with those two treatments only mediating approximately 20% of the already modest total effect [16,18].
PACE also excluded the sickest patients, (those unable to attend a study centre, potentially 25% of the patient population [19]).
Longer term follow-up data has not yet been reported, so we do not know if even these methodologically problematic and clinically marginal results are sustained.
Lastly, probably the two most comparable contemporary trials to PACE (Núñez, and FINE - "the sister study to PACE") failed to support its therapeutic claims [20,21,22].
Collectively these findings do not suggest that the CBT-GET based model employed by PACE has demonstrated sufficient explanatory or therapeutic power upon which to safely and productively build clinical and research programs. If anything they have confirmed the paucity of that model, and the increasingly urgent need to look elsewhere for answers to this troubled, refractory, and distressing condition.
Prof. Stephen Holgate, chair of the UK CFS/ME Research Collaborative, gave a keynote speech recently in which he described our current understanding of this condition as "bathing... ...in ignorance" [23]. I can only agree.
References
1. Lynch S. Re: PACE trial authors’ reply to letter by Kindlon. http://www.bmj.com/content/347/bmj.f5963/rr/671093
2. Lynch S. Measures need to capture patients’ views and experiences more effectively. BMJ 2013;346:f1553. http://www.bmj.com/content/346/bmj.f1553
3. Wiborg JF, et al. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity.
Psychol Med. 2010 Aug;40(8):1281-7. doi: 10.1017/S0033291709992212.
4. White PD, Goldsmith KA, Johnson AL, et al, on behalf of the PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial.
Lancet, 2011; published online Feb 18. DOI:10.1016/S0140-6736(11)60096-2.
5. Wise RA, Brown CD. Minimal clinically important differences in the six-minute walk test and the incremental shuttle walking test.
COPD, 2005; 2(1): 1259.
6. Enright PL, Sherrill DL. Reference equations for the six-minute walk in healthy adults.
Am. J. Respir. Crit. Care Med., 1998; 158: 13841387
7. Lipkin DP, Scriven AJ, Crake T, Poole-Wilson PA. (1986) Six minute walking test for assessing exercise capacity in chronic heart failure.
Br Med J (Clin Res Ed). 292:653-5.
8. Ross RM, Murthy JN, Wollak ID, Jackson AS. The six minute walk test accurately estimates mean peak oxygen uptake.
BMC Pulmonary Medicine, 2010, 10: 3
9. VanNess JM, et al. Postexertional malaise in women with chronic fatigue syndrome. J Womens Health (Larchmt). 2010 Feb; 19(2): 239-44.
10. Light AR, et al. Moderate exercise increases expression for sensory, adrenergic, and immune genes in chronic fatigue syndrome patients but not in normal subjects.
J Pain 2009;10:1099
11. Ng SS, et al. Walkway Length, But Not Turning Direction, Determines the Six-Minute Walk Test Distance in Individuals With Stroke.
Archives of Physical Medicine and Rehabilitation: Volume 92, Issue 5 , Pages 806-811, May 2011
12. Sciurba F, et al. Six-minute walk distance in chronic obstructive pulmonary disease: reproducibility and effect of walking course layout and length.
Am J Respir Crit Care Med. 2003 Jun 1;167(11):1522-7. Epub 2003 Feb 20.
13. White PD, Sharpe MC,Chalder T, DeCesare JC and Walwyn R for the PACE trial group. Protocol for the PACE trial: A randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy.
BMC Neurology 2007, 7:6. doi:10.1186/1471-2377-7-6
14. Author's reply to comments in the letters section for reference 13 above:
http://www.biomedcentral.com/1471-2377/7/6/comments#306608 (Accessed 30 Jan 2014)
15. McCrone P, Sharpe M, Chalder T, et al. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis.
PLoS One. 2012;7(8):e40808.
16. P. D. White, K. Goldsmith, A. L. Johnson, T. Chalder and M. Sharpe (2013). Recovery from chronic fatigue syndrome after treatments given in the PACE trial.
Psychological Medicine, 43, pp 2227-2235 doi:10.1017/S0033291713000020
17. Cella M, Sharpe M, Chalder T. Measuring disability in chronic fatigue syndrome: reliability and validity of the Work and Social Adjustment Scale.
J Psychosom Res. 2011 Sep;71(3):124-8. doi: 10.1016/j.jpsychores.2011.02.009.
18. Goldsmith K, et al. How do treatments for chronic fatigue syndrome work? Exploration of instrumental variable methods for mediation analysis in PACE – a randomised controlled trial of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care.
Trials 2011, 12(Suppl 1):A144 doi:10.1186/1745-6215-12-S1-A144
19. CFS/ME Working Group. Report to the Chief Medical Officer of an Independent Working Group. London: Department of Health; Jan 2002.
http://www.erythos.com/gibsonenquiry/docs/cmoreport.pdf (Accessed 30 Jan 2014)
20. Núñez M, et al. Health-related quality of life in patients with chronic fatigue syndrome: group cognitive behavioural therapy and graded exercise versus usual treatment. A randomised controlled trial with 1 year of follow-up
Clin Rheumatol. 2011 Mar;30(3):381-9. doi 10.1007/s10067-010-1677-y
21. Wearden AJ, et al. Nurse led, home based self help treatment for patients in primary care with chronic fatigue syndrome: randomised controlled trial.
BMJ 2010;340:c1777. doi: 10.1136/bmj.c1777.
22. PD White
http://www.ico.org.uk/~/media/documents/decisionnotices/2013/fs_50463661... (Accessed 30 Jan 2014)
23. Prof S. Holgate:
http://www.actionforme.org.uk/get-informed/research/our-research-related... (Accessed 30 Jan 2014)
Competing interests: No competing interests