Intended for healthcare professionals

Rapid response to:

Analysis And Comment Public health

Influenza vaccination: policy versus evidence

BMJ 2006; 333 doi: (Published 26 October 2006) Cite this as: BMJ 2006;333:912

Rapid Response:

Call to order: assessing the evidence

By assessing the evidence for influenza vaccination, Thomas Jefferson
started a discussion that seems to demonstrate some lack of common
understanding of important concepts in empirical research that will make
it difficult for the scientific community to interact with policy makers
in a coherent way.

Much of the debate is around the insights (or epistemiological
merits) that can be expected from (the results of) randomized controlled
trials (RCT). It should be clear that Karl Popper's falsification paradigm
for empirical research provides the rationale, and RCT is only a technique
for its implementation: Causal inference must exclude all other possible
causes that could explain the observed treatment effect, and the RCT is a
convincing design, then. However, a RCT cannot accomplish more than a test
of the concept (efficacy), and it can never replace a study of
effectiveness, on any logical grounds. As with car racing, in which a
care maker gives evidence of knowing how to make cars (the proof of
concept) while the real-life (market) setting is every-day car driving in
the streets, RCT efficacy may be considered sufficient empirical evidence
that the test compound works in the test stand; a positive efficacy trial
may also justify regulatory approval and market entry, but by no logical
means would the RCT give re-assurance of success in the target population,
as long as a logical basis for extrapolating from the test setting is

When it is commonly understood that efficacy is a research goal, and
effectiveness is the health service objective, when it is accepted that
proof of concept (efficacy) is necessary, but must not be substituted for
proper effectiveness studies, then no dissent about the value of currently
available evidence will remain: it does not seem compelling, as Thomas
Jefferson pointed out.

Competing interests:
None declared

Competing interests: No competing interests

23 November 2006
Jochen Mau
Professor and Director
Institute of Statistics in Medicine, Heinrich Heine University Hospital, 40001 Duesseldorf