Using quality of life measures in the clinical setting

Although we read with interest the article by Higginson and Carr [1],
we see a serious cause for concern in what it seems to be the perpetuation
of the belief that responsiveness is an independent property needed by
quality of life (QoL) measures used in clinical practice.

To our knowledge, this credence has its roots in a non-proved
assumption stated in the 80’s by a group of authors from the McMaster
University [2,3]. The group mainly sustained the responsiveness definition
in a three-type compartmentation of QoL tests: discriminative (used to
distinguish between individuals), predictive, or evaluative (used to
measure longitudinal change) [2]. While demonstration of reliability and
validity would be sufficient for concluding that an instrument is useful
for description or prediction, instrument’s responsiveness (or the ability
to detect important clinical changes) was assumed to be necessary for
evaluation [2,3]. Despite being mere speculation, this opinion has become
very popular in the medical literature.

Setting aside this tripartite framework has been previously
criticised because of its oversimplified nature [4], we would like to go a
step further and put in doubt the entire responsiveness thesis.

According to principles of test theories [5], the responsiveness
assessment is nothing more than a standard validation procedure embedded
in a longitudinal design.

Validity is an overall evaluative judgement of the degree to which
empirical evidence and theoretical rationales support the adequacy of
interpretations on the basis of test scores. Conventional approaches to
validate a test are based on studies of expected performance differences
over time, across groups and settings, and in response to experimental
treatments and manipulations [6].

There are many sources of invalidity [5]. In our case is important to
be aware of the fact that markedly skewed distributions of scores (floor
effects referred by Higginson and Carr on the BMJ’s website), jeopardise
the validity of any test interpretation without regard of the research
study design (cross-sectional or longitudinal).

Imagine that you have a thermometer that only measures from 100.4 to
122 degrees F and some patients suffering fever from 102.2 to 104. You
will be capable of discriminating temperatures between subjects, but
unable to detect “clinically” important individual changes if all of them
return to a “normal” temperature around 96.8. It could be said that the
instrument was “valid” for discrimination but not “responsive” to detect
clinically significant longitudinal differences. If patients had moved
their temperature within the limits of the thermometer range, the
instrument would have been responsive to changes. The truth is that the
instrument is ONLY “valid” (and by extension “responsive”) in the range
from 100.4 to 122, but completely “invalid” (and by extension
“unresponsive”) below 100.4.

Responsiveness is no more than a new name for an old friend.

Prieto L1, Sacristan JA1, Casado A2, Gómez JC1

1. Spanish Group for the Study of Methodology in Clinical Research.
Madrid, Spain.

2. Iberoamerican Cochrane Centre. Barcelona, Spain.

References

1. Higginson IJ, Carr AJ. Using quality of life measures in the
clinical setting. BMJ 2001; 322: 1297-300.

2. Kirshner B, Guyatt G. A methodological framework for assessing health
indices. J Chron Dis 1985; 38: 27-36.

3. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the
usefulness of evaluative instruments J Clin Epidemiol 1987; 40: 171-8.

4. Williams JI, Naylor CD. How should health status measures be assessed?
Cautionary notes on Procrustean frameworks. J Clin Epidemiol 1992; 42:
1347-51

5. Nunnally JC, Bernstein IH. Psychometric theory (3rd ed.). New York:
McGraw-Hill, 1994.

6. Messick S. Validity of psychological assessment: validation of
inferences from persons' responses and performances as scientific inquiry
into score meaning. Am Psychologist 1995; 50: 741-9.