Pragmatic randomised trials using routine electronic health records: putting them to the test

BMJ 2012; 344 doi: (Published 7 February 2012)
Cite this as: BMJ 2012;344:e55
  1. Tjeerd-Pieter van Staa, head of research and honorary professor of epidemiology123,
  2. Ben Goldacre, research fellow3,
  3. Martin Gulliford, professor of public health4,
  4. Jackie Cassell, professor of primary care epidemiology5,
  5. Munir Pirmohamed, NHS chair of pharmacogenetics6,
  6. Adel Taweel, senior lecturer in software engineering4,
  7. Brendan Delaney, Guy’s and St Thomas’ charity chair in primary care research4,
  8. Liam Smeeth, professor of clinical epidemiology3
  1. 1General Practice Research Database (GPRD), Medicines and Healthcare products Regulatory Agency, 151 Buckingham Palace Road, London SW1W 9SZ, UK
  2. 2Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, the Netherlands
  3. 3London School of Hygiene and Tropical Medicine, London, UK
  4. 4Department of Primary Care and Public Health Science, King’s College, London, UK
  5. 5Division of Primary Care and Public Health, Brighton and Sussex Medical School, University of Brighton, Brighton, UK
  6. 6Wolfson Centre for Personalised Medicine, Institute of Translational Medicine, University of Liverpool, Liverpool, UK
  1. Correspondence to: T P van Staa Tjeerd.vanstaa{at}
  • Accepted 23 November 2011

What to prescribe for a patient in general practice when the choice of treatments has a limited evidence base? Tjeerd-Pieter van Staa and colleagues argue that using electronic health records to enter patients into randomised trials of treatments in real time could provide the answer


Ten years ago, in a paper called Britain’s Gift, the editor of the BMJ at the time and the director of the UK Cochrane Centre outlined a vision of medicine for the 21st century: easy access to good quality reviews of clinical evidence, and the streamlined recruitment of patients into randomised trials as a matter of routine whenever there is uncertainty about choice of treatment.

“For example,” they explained: “we still do not know which treatments are useful for acute stroke, but if every patient in the world experiencing a stroke were admitted to trials we would have enough patients within 24 hours to answer many of these questions.”1

The first goal of easy access to good quality reviews of evidence is on its way to being realised. Trials, however, remain exceptional in everyday clinical care, and sometimes address comparisons that are irrelevant to doctors and patients because they compare new treatments with placebo rather than with the best treatments currently available. Furthermore, trials are often conducted in idealised or unrepresentative patient groups.2 Because of these problems, randomised trials commonly fail to inform decisions in everyday clinical care: they address the abstract question of an intervention’s efficacy under ideal conditions, rather than its effectiveness when used in usual clinical practice, on outcomes that are important to patients.3 4

Here we describe a UK project to implement randomised trials as unobtrusively as possible in the everyday clinical work of general practitioners (GPs), comparing treatments that are already in common use, and using routinely collected electronic healthcare records (EHR) both to identify participants and to gather results. We discuss the rationale for this approach, the potential for improving clinical evidence at low cost, and the barriers encountered.

Opportunities for using EHR data for randomised trials

Reports from both the Council for Science and Technology5 and from the Academy of Medical Sciences6 in 2005 and 2006 highlight the potential of EHR data for translational health research, and research with EHR data has been recognised as a key activity in the Department of Health’s national health research strategy.7 Healthcare records are routinely stored on computers in UK general practice (most people in the UK are registered with a general practitioner). Some GP databases can now be linked anonymously to other healthcare datasets, including hospital admissions records, death certificates, and disease registries. This record linkage system has been implemented within the general practice research database (GPRD) used in the trials presented here, and could be implemented more widely. It allows long term, anonymous, unobtrusive follow-up for major clinical outcomes, at low cost, and with no extra time burden for the clinician, health service, or patient.

Conventional trial recruitment is often problematic, with many trials failing to meet their recruitment targets.8 The EHR database may also be used to recruit patients into trials: it is searched to compile a list of potentially eligible patients, which is sent securely to the clinician’s desktop computer. When a patient on the eligibility list attends the practice for events related to the trial, a flag appears on screen to notify the clinician that the patient may be eligible for recruitment, with a link to the study website. If patient and clinician agree to participate and the GP confirms eligibility, the patient is randomised.

Table 1 outlines the research questions, interventions, and measurements in the first two feasibility trials for the randomised evaluations of accepted choices in treatment (REACT) trials that we are initiating. For these projects, there are daily downloads of GP EHR into the GPRD. The trial database can be compared periodically to the full research database for fraud detection and generalisability of the randomised population.

View this table:
Table 1

Research questions, interventions, and measurements in two feasibility REACT trials initiated within the GPRD

A key requirement for the REACT trials is that so called usual conditions apply as far as possible. Application of usual conditions is important to ensure external validity, and also to promote recruitment and retention: incompatibilities between the protocols for randomised trials and usual clinical practice can act as a barrier to recruitment.8 The studies we have outlined are open label and non-blinded, with patients’ progress monitored as usual in clinical practice, and these follow-up data extracted from the EHR. The only added feature in pragmatic randomised evaluations is that, among currently accepted treatments where there is no evidence on comparative effectiveness, treatment choices are based on random allocation rather than on arbitrary decisions by clinicians.

Such uncertainty in choice of treatments remains common. For many treatments in common use there is no evidence to inform choice between available options, and, for most new medicines, evidence based assessment of any added therapeutic value is not published at the time of market authorisation.9 The UK database of uncertainties about the effects of treatments (DUETs) was established to publish uncertainties about the effects of treatments that cannot currently be answered by referring to reliable, up to date, systematic reviews of existing research evidence.10 Where there is no evidence comparing two commonly used treatments for the same condition, clinicians and patients have no way of knowing which is the more effective. In these circumstances, treatments are chosen arbitrarily, through a non-scientific, haphazard process. Treating patients in this arbitrary manner generates no new evidence to improve clinical practice.

Where there is no evidence to guide the decisions of doctors and patients, it is ethically acceptable and actively desirable to offer willing patients the option of randomisation to assess which treatment is preferable. General Medical Council guidance requires doctors to resolve uncertainties about the effects of treatments,11 and good medical practice requires that doctors communicate evidence clearly to patients. Randomisation with systematic data collection is the most rational and ethical way to resolve uncertainties.12 13 Embedding randomised evaluations within usual clinical practice can achieve this goal, and increase the likelihood that clinicians will declare honestly to their patients when there is uncertainty about the relative merits of alternative treatment options.

In principle, as an ultimate goal, in every situation where there is genuine uncertainty about which of two or more widely accepted treatments is best, all willing patients could be offered randomisation as part of routine clinical care, and their progress followed up through EHR. If comparisons of accepted treatments could be conducted within the routine clinical practice setting in this fashion, the benefits would be considerable in terms of new evidence, and cost effectiveness in research.14

Challenges with using EHR data for randomised trials

The REACT approach does, however, face substantial challenges (table 2 lists opportunities and challenges, with strategies to address them, for REACT trials conducted within EHR databases). Firstly, and most importantly, are current norms in research governance. The requirements for informed consent and regulatory oversight in all trials are time consuming and expensive, even for trials comparing two interventions that have already been shown to be safe and are in widespread and routine use. This is a problem that has raised concerns over almost two decades.15 Clinicians who admit there is uncertainty in a choice between two interventions, and wish to address the uncertainty by offering treatment in the context of a randomised evaluation, are subject to intense regulatory scrutiny. Yet during routine clinical care—in situations where there is no comparative effectiveness research to guide treatment choice, so that decisions are equally arbitrary—no such constraints apply.12 Experimentation by politicians on the delivery of health services also does not suffer from this intense regulatory scrutiny.16

View this table:
Table 2

Potential opportunities and challenges with REACT trials conducted within EHR databases

Several justifications have been suggested for so called research exceptionalism—the phenomenon whereby more stringent rules are applied to research than to usual clinical practice, even for treatments in widespread use. One frequently raised justification is that research does not in general specifically aim to benefit the participants, but rather to generate knowledge; study participants may take the risks while others accrue the benefits.17 However, this justification often does not apply to patients with chronic conditions, whose treatment next year may well benefit from knowledge gained in the randomised evaluations they participate in today. It is also unclear how a trial presents extra risk, where the randomisation is between two routine treatments already in widespread use, and with no evidence presently available to inform a choice between them. This asymmetrical approach to regulation can be traced back to the establishment of the Declaration of Helsinki18 following the Nuremberg war crimes trials.16 Informed consent is fundamental to medical ethics, but the regulations designed to prevent abuse were never intended to prevent evaluation of safety in routine practice.

In the REACT trials no new risks are introduced, but the alternative—the current situation—has demonstrable ethical problems. Good quality evidence to improve patient care cannot be reliably generated from arbitrary treatment decisions made in usual clinical practice, and patients may continue to suffer through being exposed to interventions that are later found to be inferior. Furthermore, where there is uncertainty clinical decisions are often made without fully acknowledging their arbitrariness to patients.

The extent to which research exceptionalism will restrict the benefits of the REACT trials is not yet clear. All consent forms currently cover a great deal of information normally not provided when the same treatments are routinely given outside a randomised evaluation, and are extremely time consuming to complete. UK government guidelines presently recognise that one size will not fit all with respect to the information that is required to make autonomous decisions.19 However, the guidelines also state that all randomised trials must comply with the good clinical practice quality standard, and this includes a list of 22 different topics to be covered in the information sheet.20 Research ethics committees may further add to this barrier, often including idiosyncratic administrative requirements, such as a duty on participants to inform a private health insurer (as happened in our trials).

We have concerns over these barriers to research on routine treatments, which will reduce recruitment of clinicians and patients. Requirements for informed consent should ideally be based on empirical evidence on what kind of process best informs participants, and be designed in collaboration with patients. However, the good clinical practice quality standard, which has come to be viewed as canonical, was based on expert opinion, and has little empirical evidence. A systematic review has found that evidence for the optimal amount of information to enhance patient understanding is inconclusive and limited.21 UK government guidelines state that “any researcher is faced with considerable difficulty” in selecting information for informed consent, “given the disagreement on how much information potential participants in research want.”19 There are also frank contradictions. For example, the guidelines recommend that draft versions of patient information sheets should be passed to patients in disease support groups for comments. But the same guidelines also require that all informed consent procedures in trials must adhere to current good clinical practice requirements,19 which mandate extensive content,20 so support groups are prevented from reducing information overload, for example, should they recommend this in their comments.

This extra burden may reduce recruitment and retard research throughout clinical medicine. The largest review conducted on strategies to improve trial participation found that concerns about extra effort and workload are barriers to recruitment for both doctors and patients.22 The review also recommended that trials should be framed and organised in ways that minimise differences between research and clinical practice, using simple and clear entry criteria, and address questions of clear relevance to clinical practice.22

A second major challenge of using EHR data for trials is data quality, which is of paramount importance. The REACT trials will not be suited to evaluating every type of research question. A study requiring detailed, study specific data collection at regular intervals may not be best suited to a trial using EHR data. There may also be specific outcomes that a trialist would prefer to measure that are not routinely collected in EHR. However, by definition many major outcomes are recorded in routine medical notes. Furthermore, mortality and other major clinical outcomes can now also be measured in EHR databases, and then verified across several other data sources. As an example, a heart attack in the REACT trials can be measured in the GPRD, linked hospital data, disease registries, and death certificates (if fatal). Where there is doubt, the patient’s clinician can also be asked to confirm a diagnosis for an outcome of interest.

Current progress of REACT trials

The first two REACT trials have recently been approved by the research ethics committee, along with a qualitative study that seeks feedback from GPs and patients to assess and improve the implementation. The main point of discussion by the committee was our proposed consent form. The original one page patient information draft submitted for ethical review was considered by the committee to be the “skimpiest ever” and missing much of the standard clinical trial information (the informed consent template of the UK National Research Ethics Service lists a large number of items to be covered). We resubmitted the patient information sheet, now twice the length and amended to meet only the minimum ethics committee requirements. It could be argued that the single most important consideration for informed consent in the REACT trials should be the replacement of clinician’s uncertainty with randomisation. After all, the patient could have received any of the interventions in a REACT trial by consulting another clinician.

The IT system has taken considerable time to develop and is currently undergoing testing. Once implemented, the system will provide instantaneous trial recruitment and daily analysis of EHR data that can easily be adapted to future studies. GP recruitment is also continuing. Of English GPRD practices, 42% approached have expressed interest and 15% declined (recruitment in Scotland has just started). Together with recruitment by the Primary Care Research Network, we now have over 200 interested practices, and study details (protocol and contract) have now been sent to practices.

The main challenge now will be to obtain research and development approvals from the 150 local NHS bodies that cover UK general practices. Our experience with a GPRD cluster trial23 and pharmacogenetic study found that this takes enormous effort and time, even for low risk studies, replicated at multiple sites, and often with differing systems. Our goal of maximising representativeness is achieved by recruiting at multiple sites with few patients in each, so such fragmented and diverse local administrative systems present a challenge. Finally, the project also has undertaken a review on how best to comply with the good clinical practice quality standard, since this dispersed model with electronic data collection means that site visits for scrutiny are of very limited value.


EHR databases contain a wealth of information, and their utility for randomised evaluations should be fully exploited. A revolution is long overdue in the technical and research governance frameworks for testing widely used interventions whose relative merits are unknown. Narrowly restricted studies with questionable external validity need not be the norm. Our suggestion for large scale randomisation in usual clinical practice may face several challenges, some of them technical, but most of them related to research governance procedures. We hope that these barriers will be overcome, by providing proof of concept for a streamlined simple framework for undertaking REACT trials, in which recognition of widespread uncertainties about the effects of treatments will motivate clinicians and patients to participate in randomised evaluations as a matter of routine.


Cite this as: BMJ 2012;344:e55


  • Contributors and sources: This article arose from discussions of the design of protocols and systems for REACT trials. TS, BG, MG, JC, MP, BD, and LS contributed to the discussions about the methods and potential role within clinical research for the REACT trials. AT, BD and TS contributed to the development of the information systems behind REACT trials. The acronym REACT was suggested by Iain Chalmers, chair of the trial steering committee. TS and LS are guarantors.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at (available on request from the corresponding author) and declare: the two feasibility REACT trials within the general practice research database (GPRD) are being funded by the Wellcome Trust and the National Institute for Health Research Health Technology Assessment. The GPRD is owned by the UK Department of Health and operates within the Medicines and Healthcare products Regulatory Agency (MHRA) and has received funding from the MHRA, Wellcome Trust, Medical Research Council (MRC), National Institute for Health Research (NIHR) Health Technology Assessment programme, Innovative Medicine Initiative, UK Department of Health, Technology Strategy Board, Seventh Framework Programme EU, various universities, contract research organisations, and pharmaceutical companies. The Department of Pharmacoepidemiology and Pharmacotherapy, Utrecht Institute for Pharmaceutical Sciences has received unrestricted funding for pharmacoepidemiological research from GlaxoSmithKline, Novo Nordisk, the private-public funded Top Institute Pharma, the Dutch Medicines Evaluation Board, and the Dutch Ministry of Health. MP is supported by the Department of Health, MRC, NIHR, and the Wellcome Trust. LS is an NIHR senior investigator and is supported by a senior clinical fellowship from the Wellcome Trust. BG is supported by a fellowship from the Wellcome Trust. They have no other relationships or activities that could appear to have influenced the submitted work.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: and