Letters Chronic fatigue treatment trial

PACE trial authors’ reply to letter by Kindlon

BMJ 2013; 347 doi: https://doi.org/10.1136/bmj.f5963 (Published 15 October 2013) Cite this as: BMJ 2013;347:f5963

Re: PACE trial authors’ reply to letter by Kindlon

There were many major faults in the published analysis of the PACE trial data: numerical, statistical and logical. Even far back in the days of John Snow and his study of cholera, or Ignaz Semmelweis and puerperal fever, a good grasp of statistics has been essential.

Sometimes it is difficult to realise the magnitude of these errors, so let me just illustrate one of them – the use of the mean and variance to determine boundaries.

The SF-36 Physical Functioning scale is a 21 point scale, where each point is multiplied by 5 to give a score out of 100 (0 to 100 inclusive). It is designed to measure difficulty in functioning, as opposed to measuring the range of physical function in healthy folk, so when applied to a random sample of adults there is a heavy bias, or clumping effect, at the maximum, healthy score of 100. In such a biased distribution, statisticians use the middle value, or median, to determine the "average" value, and for the SF-36 physical function it would be 100. Over 80% people of working age without long-term illness would have scores of 90 or more.

To define normal function, the authors of the PACE trial used normative data from a study by Bowling[1] which warned of a heavily clumped (or skewed) distribution of scores. PACE decided to set the boundary for normal function at a score of 60 or above, using the mean (84) minus one standard deviation (s.d.) of 24.

As a comparison, the distribution for incomes in the UK in 2007/08[2] has comparable measure of bias, a mean of £26800 and a standard deviation of £29500. Using mean minus s.d. gives us a "normal" income of anything above minus £2700, which is as inappropriate to income as the SF-36 target score of 60 is to normality/recovery. There can be no statistical justification for accepting that the mean minus s.d. here somehow defines normality.

I believe that the results of the PACE study are indeed significant, but that its evidence clearly shows that the default assumption of the importance of psychological factors in ME/CFS is unjustified.

If you would like to look at a short summary of numerical, statistical and logical errors, here is a link: http://evaluatingpace.phoenixrising.me/aps2furtherfault.html

A group of us with mathematical and scientific backgrounds have also produced a much deeper critique of the PACE study, which may be of interest: http://evaluatingpace.phoenixrising.me/homepageanim.html

Or there is a more visual presentation of the basic ideas here: http://www.youtube.com/user/MEAnalysis/videos

[1] Ann Bowling; Short Form 36 Health Survey http://jpubhealth.oxfordjournals.org/content/21/3/255.full.pdf

[2] The data from Wikipedia (http://en.wikipedia.org/wiki/Income_in_the_United_Kingdom) is simpler to follow than that of the Office For National Statistics.

Competing interests: No competing interests

21 November 2013
Graham McPhee
Retired
n/a
Sussex
Click to like:
117