CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials

In their discussion on the CONSORT guidelines for clinical trial
reporting, Moher et al. state that accurate and transparent reporting are
important (1). About blinding, they write as follows: “Participants may
respond differently if they are aware of their treatment assignment ...
These biases have been well documented” (6 citations).

One of the six cited papers is the trial by Karlowski et al. (2)
which reported that the benefit of vitamin C against the common cold was
explained by the break in the blind. In the abstract: [the vitamin C]
“effects demonstrated might be explained equally well by a break in the
double blind” (2). The Karlowski trial has frequently been cited by
clinical trialists as an example of the importance of blinding, and by
specialists in nutrition and infectious diseases as an evidence that
vitamin C does not have a real effect on the common cold (3,4).

Given that Moher et al. argue for the importance of good quality
reporting of controlled trials, it is surprising that they ignore obvious
shortcomings in the Karlowski paper. The trial report is inconsistent with
several of the CONSORT 2010 items (1).

Item 1b. Abstract : “The abstract should accurately reflect what is
included in the full journal article” [in their text] and “for the primary
outcome, a result for each group and the estimated effect size and its
precision” [in Table 2]. Karlowski et al.'s abstract does not describe the
results for the primary outcome. Their results section describes:
“Volunteers taking placebo had colds of a mean duration of 7.14 days,
while those taking 3 gm of ascorbic acid (groups 2 and 3) had colds of a
mean duration of 6.59 days and those taking 6 gm had colds of a mean
duration of 5.92 days. Thus, each 3-gm increment of ascorbic acid would
appear to shorten the mean duration of a cold by approximately half a
day”, but this is not mentioned in their abstract (2). Instead, in their
abstract Karlowski et al. describe the findings of a post hoc subgroup
analysis based on guessing the treatment, even though half of recorded
common cold episodes were missing from the subgroup analysis without any
explanation.

Item 3b. “Important changes to methods after trial commencement (such
as eligibility criteria), with reasons” (1). Although Karlowski et al. did
motivate their post hoc subgroup analysis, they did not give any details
about the groups who were “blinded” and “unblinded” after the trial.
Karlowski administered vitamin C in two ways using 2x2 factorial design:
prophylactically each day over the study and therapeutically for 5 days
when a participant caught a cold. Thus, a participant can be “unblinded”
for either or both of the supplementation methods. However, Karlowski et
al. did not report which supplementation method the “unblinded”
participants guessed correctly.

Item 12b. “Methods for additional analyses, such as subgroup analyses
and adjusted analyses. ... Because of the high risk for spurious findings,
subgroup analyses are often discouraged. Post hoc subgroup comparisons
(analyses done after looking at the data) are especially likely not to be
confirmed by further studies. Such analyses do not have great credibility”
(1). In the Karlowski trial, the subgroup analysis by “guessing the
treatment” was a post hoc subgroup comparison. The methods of the subgroup
analysis are poorly described, see Items 3b and 13b. Furthermore,
Karlowski et al. used “correct answer” as a surrogate for “knowing”
without considering that many answers we correct purely by guesswork.
Thus, the main finding of the Karlowski study, on the basis of their
abstract - the reason for Moher et al. to cite the study (1) - is based on
a poorly described post hoc subgroup analysis.

Item 13b. “For each group, losses and exclusions after randomisation,
together with reasons” (1). In the Karlowski trial, 42% (105/249) of
recorded common cold episodes were missing from the subgroup analysis (4),
but no reasons are given to this exclusion. Furthermore, the maximum
effect of vitamin C on common cold duration was even greater in the
“missing group” than in the entire study population (4), but this was not
commented on by Karlowski et al.

Item 20. “Trial limitations, addressing sources of potential bias,
imprecision, and, if relevant, multiplicity of analyses” (1). In the
abstract, the main finding of the Karlowski trial was the subgroup
difference in the effect of vitamin C by guessing the treatment. However,
the shortcomings of the subgroup analysis are not properly discussed.
There are numerous logical inconsistencies that should have been noted by
the original authors (3-5).

The Karlowski trial is particularly important for two reasons: for
methodological and for biological reasons. First, it has been and still is
used as an example of the “placebo effect in action” (1,3). Second, the
Karlowski trial is a frequent citation in medical textbooks when stating
that vitamin C is ineffective against the common cold (3), whereas placebo
-controlled trials have shown quite consistently that it is effective,
even though the practical significance is unsettled (8).

Over a decade ago, I pointed out the problems of the Karlowski
subgroup analysis (4). The principal investigator of the Karlowski trial
did not find errors in my reanalysis (6,7). The study cannot be considered
as an example of placebo effect in action.

Thus, while Moher et al. propose the CONSORT items as a guide for
authors who write trial reports and for the readers of such reports, they
ignore those items when they refer to the Karlowski trial as an evidence
justifying their claim that “biases [caused by the awareness of their
treatment assignment] have been well documented” (1).

Furthermore, Moher et al. do not refer to a recent large meta-
analysis of 202 trials with 60 clinical conditions, which directly
compared a placebo arm with a no-treatment arm (9). Most of the included
trials had three arms so that the third arm received an active
intervention. Therefore, the participants of the placebo arms did not know
whether they were given the active treatment or not (blinded to the
treatment), whereas the no-treatment participants knew that they were not
being treated. Thus, this meta-analysis measures the importance of
blinding. Placebo had no effect when the outcome was binary or objective,
whereas it had effects on several subjective and continuous outcomes (9).
Furthermore, this meta-analysis gives estimates for bias potentially
caused by the lack of blinding in trials on various clinical conditions.
It would seem much more useful to consider the specific conditions when
the lack of blinding may cause substantial bias, in contrast to universal
statements such as “biases have been well documented (1)”.

Citation bias means that authors refer to studies that are consistent
with their preconceptions (10). There is evident citation bias in the
article by Moher et al. (1). When arguing that the lack of blinding causes
bias in controlled trials, they refer to an old study which supports their
preconceptions (2), ignoring the evidence which indicates that the old
study was erroneously analyzed (3-5). In addition, they ignore an
extensive meta-analysis which analyses the effect of blinding on 60
clinical conditions (9). Although trial reports should describe the level
of blinding, bias caused by the awareness of treatment assignment has not
been “well documented”.

Harri Hemilä
University of Helsinki
harri.hemila@helsinki.fi

REFERENCES

1. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux
PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and
elaboration: updated guidelines for reporting parallel group randomised
trials. BMJ 2010;340:c869. doi: 10.1136/bmj.c869.

2. Karlowski TR, Chalmers TC, Frenkel LD, Kapikian AZ, Lewis TL, Lynch JM.
Ascorbic acid for the common cold: a prophylactic and therapeutic trial.
JAMA 1975;231:1038-42.

3. Hemilä H. The most influential trial on vitamin C and the common cold:
Karlowski et al. (1975). In: Do vitamins C and E affect respiratory
infections? [PhD Thesis] University of Helsinki, Helsinki, Finland,
2006:21-7. Available at:
http://ethesis.helsinki.fi/julkaisut/laa/kansa/vk/hemila/ and
http://www.ltdk.helsinki.fi/users/hemila/karlowski/

4. Hemilä H. Vitamin C, the placebo effect, and the common cold: a case
study of how preconceptions influence the analysis of results. J Clin
Epidemiol 1996;49:1079-84.

5. Hemilä H. Analysis of clinical data with breached blindness
[commentary]. Stat Med 2006;25:1434-7.

6. Chalmers TC. To the preceding article by H. Hemilä. J Clin Epidemiol
1996;49:1085.

7. Hemilä H. To the dissent by Thomas Chalmers. J Clin Epidemiol
1996;49:1087.

8. Hemilä H, Chalker EB, Douglas RM. Vitamin C for preventing and treating
the common cold. Cochrane Database Syst Rev 2010;(3):CD000980.

9. Hrobjartsson A, Gøtzsche PC. Placebo interventions for all clinical
conditions. Cochrane Database Syst Rev 2010;(1):CD003974.

10. Gøtzsche PC. Reference bias in reports of drug trials. BMJ
1987;295:654-6.

Competing interests:
None declared