The article "Regression towards the mean", outlined some important
aspects of regression that are helpful to many. In a likewise fashion,
this response is offered.
Often medical instruments must be compared that measure across a
range when a master, high accuracy standard does not exist or can not be
used due to cost or risk. Examples include tonometers that measure
intraocular pressure under the cornea or blood pressure monitors. It is
not easy to evasively put high accuracy sensors inside the body.
For such comparisons, instruments are often tested using regression
as discussed in the main article and considered to be dissimilar (loosely
defined here) because the slope is not as close to unity as desired.
The often overlooked fact is that the ordinary least squares (OLS)
method should rarely be used in such cases. It underestimates the mean
since it minimizes the sum of squared VERTICAL distances.
When measurement error exists for both variables, it is better to
measure perpendicular distances of the points from the fitted line. Then
the slope would not be underestimated as discussed above. When making such
a device comparison, a simple shortcut exists. One can determine the slope
of Y vs X as well as the slope of X vs Y. The true slope lies somewhere
between. Orthogonal regression is just one appropriate method - others may
apply. Some titles include Deming, Type II, Geometric or Principle Axis
regression.
Though many references exist such as Bartlett (1949), Berkson (1950),
Deming (1943) and Madansky (1959) many continue to apply the incorrect
regression type for their problem.
As an aside, often when comparing two medical instruments, other
comparisons should be made as well for full understanding:
1.) Bland-Altman plot (1983): Subjective check for bias trends
2.) Maloney-Rastogi method (1970): Quantitative check for a linear
bias trend
3.) A simple paired t test: To check simple DC bias
If repeat runs (replicates) exist for each treatment (person), then a
few more tools can be explored:
4.) A paired t test of "within-treatment" variation for the two
instruments. A transform can be applied as discussed by Bland and Altman
in other articles.
5.) The pooled 'per instrument' treatment variances may be compared
using the F test. This compares the repeatability or precision in addition
to bias.
6.) The standard deviation of differences, sdiff, helps identify a
measure of difference across a range; but it does not indicate how much
variation comes from each instrument or the measured item.
7.) Maximum Likelihood Estimators (MLE's) can be used to test more
complex models and assumptions.
It should be noted that the R-squared and slope results from OLS
regression and the first three tests described above are important but the
second four tests are also needed to compare precision (if repeat runs
exist)
Both bias and precision must be checked to test two instruments for
interchangeability, practical agreement or "sameness".
Comparing precision has been discussed by Grubbs, Jaech, Lin,
Hanumara, Thompson and many others but for cases without repeat runs per
treatment.
Hopefully, this article can serve as a helpful guideline for
comparisons that need more than just the OLS regression even though the
detailed references and proofs were omitted.
OLS regression is a good tool; but it should not be alone in the
‘statistical comparison’ toolbox.
Rapid Response:
A caution when using regression
The article "Regression towards the mean", outlined some important
aspects of regression that are helpful to many. In a likewise fashion,
this response is offered.
Often medical instruments must be compared that measure across a
range when a master, high accuracy standard does not exist or can not be
used due to cost or risk. Examples include tonometers that measure
intraocular pressure under the cornea or blood pressure monitors. It is
not easy to evasively put high accuracy sensors inside the body.
For such comparisons, instruments are often tested using regression
as discussed in the main article and considered to be dissimilar (loosely
defined here) because the slope is not as close to unity as desired.
The often overlooked fact is that the ordinary least squares (OLS)
method should rarely be used in such cases. It underestimates the mean
since it minimizes the sum of squared VERTICAL distances.
When measurement error exists for both variables, it is better to
measure perpendicular distances of the points from the fitted line. Then
the slope would not be underestimated as discussed above. When making such
a device comparison, a simple shortcut exists. One can determine the slope
of Y vs X as well as the slope of X vs Y. The true slope lies somewhere
between. Orthogonal regression is just one appropriate method - others may
apply. Some titles include Deming, Type II, Geometric or Principle Axis
regression.
Though many references exist such as Bartlett (1949), Berkson (1950),
Deming (1943) and Madansky (1959) many continue to apply the incorrect
regression type for their problem.
As an aside, often when comparing two medical instruments, other
comparisons should be made as well for full understanding:
1.) Bland-Altman plot (1983): Subjective check for bias trends
2.) Maloney-Rastogi method (1970): Quantitative check for a linear
bias trend
3.) A simple paired t test: To check simple DC bias
If repeat runs (replicates) exist for each treatment (person), then a
few more tools can be explored:
4.) A paired t test of "within-treatment" variation for the two
instruments. A transform can be applied as discussed by Bland and Altman
in other articles.
5.) The pooled 'per instrument' treatment variances may be compared
using the F test. This compares the repeatability or precision in addition
to bias.
6.) The standard deviation of differences, sdiff, helps identify a
measure of difference across a range; but it does not indicate how much
variation comes from each instrument or the measured item.
7.) Maximum Likelihood Estimators (MLE's) can be used to test more
complex models and assumptions.
It should be noted that the R-squared and slope results from OLS
regression and the first three tests described above are important but the
second four tests are also needed to compare precision (if repeat runs
exist)
Both bias and precision must be checked to test two instruments for
interchangeability, practical agreement or "sameness".
Comparing precision has been discussed by Grubbs, Jaech, Lin,
Hanumara, Thompson and many others but for cases without repeat runs per
treatment.
Hopefully, this article can serve as a helpful guideline for
comparisons that need more than just the OLS regression even though the
detailed references and proofs were omitted.
OLS regression is a good tool; but it should not be alone in the
‘statistical comparison’ toolbox.
Competing interests:
None declared
Competing interests: No competing interests