Intended for healthcare professionals

Rapid response to:

Papers

Turning a blind eye: the success of blinding reported in a random sample of randomised, placebo controlled trials

BMJ 2004; 328 doi: https://doi.org/10.1136/bmj.328.74327.37952.631667.EE (Published 19 February 2004) Cite this as: BMJ 2004;328:432

Rapid Response:

The Blind leading the Blind?

The principle of blinding has become entrenched as a manoeuvre to
minimise bias in comparative trials (1, 2). An inherent part of that bias
lies in the a priori expectations of the investigators. To deny such
expectations is unrea1istic since they underpin a reasonable expectation
of equipoise that makes asking the question both necessary and possible.

Blinding is but one way of minimising bias and we would be less than
honest if we did not admit that all trials contain bias. This in turn
necessitates a determination of the direction and magnitude of the biases
and their possible impact on the interpretation of the outcome.

Ferguson and colleagues (British Medical Journal, 328: 432-4, 21
February 2004) (3) ask an important question, namely how effective is
blinding? However they may be overstating the position when they say that
if blinding is ineffective then the protections are lost, in that the
efficacy of blinding is unlikely to be a binary process, and the impact of
blinding only partly offsets the actual effect size. Thus in a large
randomised trial that demonstrates clinically significant and biologically
plausible differences in outcomes, some imperfections in blinding are
unlikely to fully explain the variance in outcome. The degree of blinding
does however add to the credibility of the reported outcome by reducing
opportunities for bias to become manifest.

Nevertheless if we advocate blinding it is important that we also ask
whether the effectiveness can be measured and if so whether differences in
that effectiveness can explain variance in the observed outcomes.

Since no validated method for measuring the effectiveness of blinding
has yet been developed it is perhaps not surprising that Ferguson et al.
found the literature wanting in this regard.

Sackett (4) raises the interesting and valid question as to whether
testing for effectiveness is merely a surrogate for testing for a priori
expectations. Therefore if the effectiveness of blinding is to be measured
it may be important to measure such expectations as explanatory variables.

Whether the effectiveness can be reliably measured or not, the steps
taken to ensure blinding should be reported as a minimum. Bias can be
applied at several levels in a trial. These include participant and
investigator expectation, outcome evaluation, analysis and interpretation.
Each of these steps is potentially subject to estimation of the impact of
potential bias. The lexicography of this has been well described (5,6).

All commentators to date (7, 4, 8, 9) have pointed out that
attempting to measure the effectiveness of blinding is essentially
confounded by outcome, whether this be the planned outcome or unintended
adverse events.

Any attempt to develop a methodology for measuring the success or
effectiveness of blinding would need to include not only the a priori
expectations but also the effects of chance. Only two trials in Ferguson’s
survey used kappa to take this into account. Other considerations are
whose estimate of the allocation is being sought and the timing of the
assessment, such as whether this was performed prior to the onset of the
intended outcome or not.

In many cases completely effective blinding is very difficult due to
known differences in adverse event profiles. Furthermore participant
expectation can be unconsciously reinforced by differing descriptions in
the consent form, which itself could be subject to blinding. Thus it is
always encouraging to note that in reported results some adverse events or
dropouts can be higher amongst placebo patients than amongst those on
active medication.

While it is difficult to remove all effects of both participant and
investigator expectation, it is possible to isolate these factors from
evaluation. Independent outcome evaluation can usually be blinded more
successfully than assessments and interactions' between participants and
investigators in the clinic.

Outcome evaluation can be blinded not only to the actual assignments
but also the nature of the investigation itself. For instance radiologists
reading films can be potentially blinded to the whole trial and be asked
merely to report on the presence or absence of a feature on an individual
assessment or to compare pairs of films.

A special case is participant reported outcome such as Fisher’s
classic taste test, as mentioned by Senn (7).

Whether blinding can be effectively measured is perhaps not the whole
question. We should be cautious about implying that failure to
unequivocally establish perfect blinding invalidates the interpretation of
the data.

In the land of the unblind, even the partially blinded may be king.

References

1. Sackett D, Haynes R, Guyatt G, Tugwell P: Clinical epidemiology. A
basic science for clinical medicine. Little Brown, Boston. 2nd Ed. 1991.

2. Altman DG, Schulz KF, Moher D, Egger M, DavidoffF, Elboume D, et
al. The revised CONSORT statement for reporting randomized trials:
explanation and elaboration. Ann Intern Med 2001 ;134: 663-94.

3. BMJ, doi:10.1136/bmj.37952.631667.EE (published 22 January 2004)

4. Sackett DL: Why we don't test for blindness at the end of our
trials. BMJ Rapid Response February 20 2004.

5. Schulz KF, Chalmers I, Altman DG. The landscape and lexicon of
blinding in randomized trials. Ann Intern Med 2002;136:254-259. BMJ Rapid
Response February 20 2004.

6. Schulz KF, Grimes DA. Blinding in randomised trials: hiding who
got what. Lancet 2002;359:696-700.

7. Senn SJ: A blinkered view of blinding. BMJ Rapid Response February
20 2004.

8. Double DB: Changing the mindset about unblinding in clinical
trials. BMJ Rapid Response February 21 2004.

9. Altman DG et al.: Testing the success of blinding and the CONSORT
statement. BMJ Rapid Response February 21 2004.

Competing interests:
Until today I believed that the robustness of the effect size could be further validated by establishing the effectiveness of the blind.

Competing interests: No competing interests

24 February 2004
Michael Goodyear
Assistant Professor
Dept. Medicine, Dalhousie University, 1278, Tower Road, Halifax, Nova Scotia, CANADA B3H 2Y9.