Effect of regression to the mean on decision making in health care
BMJ 2003; 326 doi: https://doi.org/10.1136/bmj.326.7398.1083 (Published 15 May 2003) Cite this as: BMJ 2003;326:1083
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
On “regression to the mean”
Morton and Torgerson remind us of the phenomenon of regression to the mean(1) and how it might lead to erroneous conclusions in clinical practice. They give the example of women treated for osteoporosis from Cummings et al.(2) They conclude that “Some women continue to lose bone at the first follow up measurement despite effective treatment. …because of regression to the mean, most patients (> 80%) who lost bone in the first year of treatment went on to gain bone in the second year despite no change in treatment.” Actually, Morton and Torgerson have been led to an erroneous conclusion. The figure in Cummings paper is misleading because it does not include the baseline values. The figure in Morton and Torgerson’s paper are erroneously modified from Cumming’s original figure. The modified figure now has a baseline, but show the Year 2 values incorrectly as change relative to baseline, rather than change from the Year 1 value (as correctly stated in the figure legend). A correct modification, including baseline, can be found in our 2000 paper(3) discussing the Cummings report. Actually, Morton and Torgerson have reached another erroneous conclusion: that regression to the mean applies to individual results. Instead, it applies to group data.(4) For serial bone densitometry in clinical practice, the principle of “least significant change” applies.(5) If we assume that the data points in Cummings’ initial paper on the subject represent individual and not group results, at most centers, three of the “patients” would be told they had significant increases at Year 1 and again at Year 2, four would be told there was no change at Year 1 and no change at Year 2, and one would be told there was significant loss at Year 1 but that Year 2 was no different from baseline. That data point actually represents less than 2% of the subjects in Cummings’ analysis. Regression to the mean can certainly be misleading, but can also be used to mislead!
Nelson B. Watts MD
Director, University of Cincinnati Bone Health and Osteoporosis Center
222 Piedmont Avenue, Suite 4300
Cincinnati OH 45219
phone 513-475-7400
fax 801-838-1966
voice mail 320-216-6242
e-mail nelson.watts@uc.edu
www.ucosteoporosis.com
Reference List
1. Morton V,.Torgerson DJ. Effect of regression to the mean on decision making in health care. BMJ 2003;326:1083-4.
2. Cummings SR, Palermo L, Browner W, Marcus R, Wallace R, Pearson J et al. Monitoring osteoporosis therapy with bone densitometry - Misleading changes and regression to the mean. JAMA 2000;283:1318-21.
3. Lenchik L,.Watts NB. Regression to the mean: What does it mean? Using bone density results to monitor treatment of osteoporosis. J.Clin.Densitom. 2000;4:1-44.
4. Bonnick SL. Monitoring osteoporosis therapy with bone densitometry: a vital tool or regression toward mediocrity? J.Clin.Endocrinol.Metab. 2000;85:3493-5.
5. Lenchik L, Kiebzak GM, Blunt BA, International Society for Clinical Densitometry Position Development Panel and Scientific Advisory Committee. What is the role of serial bone mineral density measurements in patient management? Journal of Clinical Densitometry. 2002;5 (suppl):S29-S38.
I can provide figures if you wish.
Competing interests:
None declared
Competing interests: No competing interests
Dear Sir
Regression to the mean is a real and important statistical
phenomenon. I think, however, that the authors may have overstated the
case for regression to the mean.
The first regression line drawn on biological data (to the best of my
present knowledge) was a plot of seed weights presented by Francis Galton
at a Royal Institution lecture in 1877. (1) Galton had seven sets of sweet
pea seeds labelled K to Q and in each packet the seeds were of the same
weight. He chose sweet peas on the advice of his cousin Charles Darwin and
the botanist Joseph Hooker as sweet peas tend not to self fertilise and
the seed weight varies little with humidity. He distributed these packets
to a group of friends throughout Great Britain who planted them. At the
end of the growing season the plants were uprooted and returned to Galton.
The seeds were distributed because when Galton had tried this experiment
himself in the Kew Gardens in 1874, the crop had failed.
He found that the weights of the offspring seeds were normally
distributed, like their parents, and that if he plotted the mean diameter
of the offspring seeds against the mean diameter of their parents he could
draw a straight line through the points - the first regression line. He
also found on this plot that the mean size of the offspring seeds tended
to the overall mean size. He initially referred to the slope of this line
as the "coefficient of reversion". Once he discovered that this effect was
not a heritable property but the result of his manipulations of the data,
he changed the name to the "coefficient of regression". This result was
important because it appeared to conflict with the current thinking on
evolution and natural selection. He went to do extensive work in
quantitative genetics and in 1888 coined the term "co-relation" and used
the now familiar symbol "r" for this value.
The phenomomen of regression to the mean is dependent on two
assumptions not mentioned in the article: ceteris paribus and the absence
of autocorrelation. Ceteris paribus is the assumption that during the
period that the data is was collected, the underlying conditions remain
unchanged. Autocorrelation is a statistical property that measures
dependency the residual values, ie the amount that the data differs from
the regression line, have on one another.
To relate the ceteris paribus assumption to the article, consider the
example of meningitis vaccination programme given. While undoubtly some of
the reduction in meningitis is due to regression to the mean, it is
difficult to agree that this assumption still holds true. The ceteris
paribus assumption can be tested for statistically with a change point
test of which there are many.
Autocorrelation may be positive or negative. When positive
autocorrelation is present the positive (those values above the regression
line) and negative residuals (those values below the regression line) tend
to occur together. When negative autocorrelation is present, positive
residuals tend to be followed by negative ones and vice versa. When
negative autocorrelation is present in a data series it can be mistaken
for regression to the mean. The presence of autocorrelation in a data
series can also be tested for.
No doubt this discussion of statistical arcana may appear irrelevant
to many of your readers. If important decisions are to be made on this
data, it seems relevant to know what the underlying assumptions are.
Reference
1. Pearson K. 1930. The Life, Labours and Letters of Francis Galton,
3A:4. Cambridge University Press.
Competing interests:
None declared
Competing interests: No competing interests
Morton
and Torgerson have pointed out a number of examples on regression to the mean
and its effects on diagnosis tests, new treatment and public health.1
We want to suggest another example on causal research. We have published a
meta-analysis of eight case-control studies on the relationship between low
density lipoprotein receptor related (LRP) protein gene exon 3 polymorphism and
sporadic Alzheimer’s disease.2
In
this letter, we present a cumulative meta-analysis on the same eight papers: we
have obtained an odds ratio with the first study, then we have added the second
published paper and we have calculated a
common odds ratio, and so on until the eight articles had been included in the
same order they were published. The results show that the first study
odds ratio was the higher (2.41) and that the meta-analysis odds ratio
progressively decreased until 1.35. Ioannidis et al have also reported that
"the first study often suggests a stronger genetic effect than is found by
subsequent studies".3
This
result would be related with both regression to the mean and publication bias.
When first studing a gene as risk factor for any disease, negative results (no
association) would be harder to publish than positive ones. This is a well-known
source of publication bias in meta-analysis. However, once a gene-disease
relationship has been published, it would be easier to publish another paper
that contradicts the first one.
Researchers
on genetic causation of diseases should have a healthy scepticism the first
time a gene-disease association is described; actually, consistency between
several reports should be required to attribute a causative paper to a gene.
References
-
Morton
V, Torgerson DJ. Effect of regression to the mean on decision making in
health care. BMJ 2003; 326:1083-4. [Free
Full Text] -
Sánchez-Guerra
M, Combarros O, Infante J, Llorca J, Berciano J, Fontalba A, Fernández-Luna
JL, Peña N, Fernández-Viadero C. Case-control study and meta-analysis of
low density lipoprotein receptor related protein gene exon 3 polymorphism in
Alzheimer’s disease. Neurosci
Letter 2001; 316:17-20. -
Ioannidis
JPA, Ntzani EE, Trikalinos TA, Contopoulos-Ioannidis DG. Replication
validity of genetic association studies. Nature Genetics 2001;
29:306-9. [Abstract]
Competing interests:
None declared
Competing interests: No competing interests
Professor Deming, the statistician and management guru credited with
Japan's post-war revival observed that "The central problem in management
and productivity is failure to understand variation." This is just as true
in clinical medicine.
An understanding of variation is as essential to practicing medicine
as the ability to take a history or elicit symptoms. In chronic disease,
failure to understand how much random variation can occur in clinical
measurements leads to constant tampering with management strategies.
Clinicians frequently overestimate the effects of advice or treatment
because of failure to appreciate regression to the mean. For example I
have often have heard primary care physicians and nurses claim that "in
their experience" patients cholesterol levels fall significantly after
receiving dietary advice. This is unlikely. With the benefit fo a control
group, it is apparent that even intensive dietary advice has a small
effect on cholesterol levels. [1] The mistake arises because we fail to
appreciate just how variable a single cholesterol measurement actually may
be. We also tend to start from a premise of belief in our beneficience,
rather than scientific scepticism. Questioning a professional's view of
their own efficacy is a fast track to unpopularity. This can be a
significant obstacle to understanding variation.
1. J L Tang, J M Armitage, T Lancaster, C A Silagy, G H Fowler, H A W
Neil 1213-1220. Systematic review of dietary intervention trials to lower
blood total cholesterol in free-living subjects BMJ 1998; 316:
Competing interests:
None declared
Competing interests: No competing interests
Regression to the mean does not explain public health effects
The article by Morton and Torgerson on the subject of regression to
the mean provides some valuable insights into this phenomenon [1].
However, we do not agree with the following statement made in the section
on public health:
‘The policy of vaccinating children against meningitis was introduced
at a time of heightened incidence. The headline benefit of a 75%-90%
reduction in cases is an overestimate as most of the reduction would have
been due to the regression to the mean.’
This statement is wrong for a number of reasons, and the benefit
mentioned is not an example of regression to the mean.
When assessing interventions applied at an individual level, such as
vaccination, it is possible to measure the direct effect of the
intervention by comparison of incidence rates in those who did and did not
receive the intervention. For the meningococcal group C vaccine the 75-90%
reductions reported [2,3] were based upon this direct effect and not on a
simple comparison of incidence before and after the vaccination campaign.
Even if the comparison was between pre and post vaccination incidence it
would still not make sense to conclude that most of the effect is due to
regression to the mean. The incidence of group C disease had risen
steadily for a number of years prior to the introduction of the vaccine.
[4] Large random annual fluctuations did not occur so there was no good
reason to expect that shortly following the introduction of vaccination
regression to some lower underlying mean level would occur that could
explain even a trivial proportion of the observed reduction.
The temporal patterns for many diseases (as well as other events)
exhibit cyclical trends, which include regular epidemics, and these
predictable changes need to be taken into account when evaluating
interventions. The random components that are the basis for regression to
the mean are often relatively small.
Regression to the mean is an important phenomenon. However, to
determine whether it provides an explanation for all or part of an
observed change, it is necessary to understand the contribution of both
the random and systematic components of variation and how they have been
allowed for.
References
1.Morton V, Torgerson DJ. Effect of regression to the mean on
decision making in health care. BMJ 2003; 326:1083-1084.
2.Ramsay M, Andrews N, Kaczmarski E, Miller E. Efficacy of
meningococcal serogroup C conjugate vaccine in teenagers and toddlers in
England. Lancet 2001;357:195-196.
3.Campbell H, Ramsay M, Andrews N, Miller E. The Impact of the
Meningococcal C Conjugate Immunisation Programme in England and Wales. New
Developments in Vaccine Research & Disease Surveillance 2003;3:1-10.
4.Miller E, Salisbury D, Ramsay M. Planning, registration, and
implementation of an immunisation campaign against meningococcal serogroup
C disease in the UK: a success story. Vaccine 2002;20:S58-S67.
Competing interests:
None declared
Competing interests: No competing interests