Intended for healthcare professionals


Experimental and observational methods of evaluation

BMJ 1994; 309 doi: (Published 20 August 1994) Cite this as: BMJ 1994;309:540
  1. N Black
  1. Department of Public Health and Policy, Health Services Research Unit, London School of Hygiene and Tropical Medicine, London WC1E 7HT.

    EDITOR, - Trevor A Sheldon's editorial perpetuates the false dichotomy between experimental and observational research.1 Once again an advocate of randomised controlled trials seeks to discredit evaluative research that uses observational data as if the two methods were alternatives rather than complementary approaches. Extremist supporters of either camp offer unacceptably simple accounts.

    Of course in some people's ideal world every intervention in the biological and social spheres would be evaluated with a randomised controlled trial. But the world is not such a utopia. As a consequence there are important roles for observational methods used with the care and rigour that, hopefully, trialists bring to their studies. What are those roles?

    Firstly, some interventions, such as defibrillation for ventricular fibrillation, have an impact so large that observational data are sufficient to show it.

    Secondly, infrequent adverse outcomes would be detected only by randomised controlled trials so large that they are rarely conducted. Observational methods such as postmarketing surveillance of medicines are the only alternative.

    Thirdly, observational data provide a realistic means of assessing the long term outcome of interventions beyond the timescale of many trials. An example is long term experience with different hip joint prostheses.

    Fourthly, whatever those who question the value of health care interventions might think, many clinicians often will not share their concern and will be opposed to a randomised controlled trial; observational approaches can then be used to show clinical uncertainty and pave the way for such a trial.

    Fifthly, despite the claims of some enthusiasts for randomised controlled trials, some important aspects of health care cannot be subjected to a randomised trial for practical and ethical reasons. Examples include the effect of volume on outcome, the regionalisation of services, a control of infection policy in a hospital, and admission to an intensive care unit. To argue that these topics could theoretically be evaluated by a randomised controlled trial is of little practical help in advancing our knowledge.

    Why advocates of trials feel the need to criticise the use of non- experimental methods is unclear. After all, randomised controlled trials have their limitations. Evaluation would benefit if everyone recognised and appreciated the vital and complementary roles that experimental and observational methods have.


    1. 1.
    View Abstract