Letters

Measurement error

A constant within subject standard deviation cannot be assumed a priori

Editor–In their statistical note about measurement error J Martin Bland and Douglas G Altman state that there is no point in estimating within subject standard deviation if we cannot assume that it is the same for all subjects.¹ This assumption, however, does not hold true in every clinically relevant situation. For example, the within subject standard deviation of many analyses is increased in patients from intensive care units. Even outside these extreme conditions, a constant within subject standard deviation cannot be assumed a priori. Homogeneity of within subject standard deviation must be verified before the estimate obtained from analysis of variance can be applied to determine whether two consecutive results in a single patient are truly different.²

When within subject standard deviation is not constant from one patient to another (presence of heterogeneity) the use of a single "mean" within subject standard deviation to estimate repeatability will underestimate the variability for several patients and will possibly lead to false conclusions about the clinical importance of the difference between two consecutive measurements made in these patients. In such cases the use of the 75th or 90th centile of within person variances across a population of similar patients has been suggested as a more appropriate estimate.³

Jacques Massé, Assistant professor in biochemistry ^a

^a Department of Biochemistry, Hôpital du Saint-Sacrement, Quebec, Canada G1S 4L8

1. Bland JM, Altman DG. Measurement error. BMJ 1996;313:744. (21 September.) [Free Full Text]
2. Fraser CG, Harris EK. Generation and application of data on biological variation in clinical chemistry. Crit Rev Clin Lab Sci 1989;27:409-37.
3. Albert A, Harris EK. Multivariate interpretation of clinical laboratory data. New York: Dekker, 1987:207-41.

Author's reply

Editor–Jacques Massé raises a good point. There are several possible causes of heterogeneity in measurement error between subjects. One is that the measurement error may be related to the level of the measurement–for example, subjects with higher values of the measurement may have more variable repeated observations than subjects with low values. Typically, we find that the standard deviation of repeated observations in the same subject is proportional to the subject's overall mean. Altman and I have discussed this possibility and an appropriate analysis.¹

It may also be that subjects in different populations have different within subject variations. A sample can tell us only about the population from which it is drawn, and when special populations such as those in intensive care are of interest it would make sense to sample them. We can then compare estimates of the within subject standard deviation between these populations. Finally, there may be heterogeneity between subjects which cannot be explained by a relation with the magnitude of the measurement. It seems to me that we should then try to explain this variation in terms of other subject variables (for example, obesity) if possible. If we cannot then I agree with Massé that a simple estimate of within subject variation could prove misleading. The method he suggests would be conservative–that is, tend to overestimate the measurement error for most patients–but this may well be preferable to underestimation.

J M Bland, Professor of medical statistics ^b

^b Department of Public Health Sciences, St George's Hospital Medical School, London SW17 0RE

1. Bland JM, Altman DG. Measurement error proportional to the mean. BMJ 1996;313:106. (13 July.) [Free Full Text]

Measurement error is that which we have not yet explained

Editor–J Martin Bland and Douglas G Altman show how to calculate measurement error for repeated measurements, using as an example peak expiratory flow rates in a group of school children.¹ We wish to point out that some of what the authors describe as error can be identified, measured, and legitimately removed as a source of variation from the within subject error. The real measurement error is therefore lower than the authors have calculated. General and specific implications are sketched.

Observe that the mean measurement in the 20 children is 299.75 l/min when they are first measured but 305.25, 315.25, and 322 l/min in subsequent trials. Thus there seems to be evidence of an upward trend, which common sense suggests is a learning effect: over the four trials the children begin to get used to using the apparatus. Consequently, some of the variability in the measurements obtained for any particular child is due to something we should not really call measurement error, since we think that we know its origins.

Performing analysis of variance, as recommended by Bland and Altman, but now including the four trials as a within subject factor, we can partition the residual variance of 27 631 found in their table 2 into two components–namely, 5958 (due to a systematic difference between trials) and 21 673 (our new residual variance) (table 1). The F ratio associated with the "trials" shows that there are indeed significant differences in measured flow rates across the four trials, though it is beyond the scope of this letter to investigate further the precise statistical description of these differences. Our newly estimated standard deviation of the measurement error is 380.22=19.50 l/min rather than 21.5, as was previously calculated.

This example illustrates an important general point. Although the statistical notion of measurement error encourages the view of random, and therefore unknowable, deviations from a "true" reading, in practice so called error may often be systematic, quantifiable, and controllable. A more encouraging orientation is that measurement error is really that which we have not yet explained. The most immediate practical advantage of reducing unexplained variance/error is that it improves the resolution of the instrument of measurement. But chipping away at the as yet unexplained variance by showing how it arises also parallels the scientific process itself: that of making the unknown known by theory building.

J R Doyle, Lecturer,^c J M Doyle, Drugs and alcohol counsellor ^d

^c School of Management, University of Bath, Bath BA2 7AY, ^d Southmead Hospital, Bristol BS10 5NB

1. Bland JM, Altman DG. Measurement error. BMJ 1996;313:744. (21 September.)