A guide to research partnerships for pragmatic clinical trialsBMJ 2014; 349 doi: https://doi.org/10.1136/bmj.g6826 (Published 01 December 2014) Cite this as: BMJ 2014;349:g6826
- Karin E Johnson, research associate1,
- Chris Tachibana, scientific editor and writer1,
- Gloria D Coronado, senior investigator2,
- Laura M Dember, professor of medicine3,
- Russell E Glasgow, associate director4,
- Susan S Huang, associate professor and medical director of epidemiology and infection prevention5,
- Paul J Martin, medical director6,
- Julie Richards, project manager1,
- Gary Rosenthal, professor7,
- Ed Septimus, medical director infection prevention and epidemiology8,
- Gregory E Simon, senior investigator1,
- Leif Solberg, associate medical director for care improvement research9,
- Jerry Suls, senior scientist10,
- Ella Thompson, project manager1,
- Eric B Larson, vice president for research1
- 1Group Health Research Institute, Seattle, WA 98101, USA
- 2Kaiser Permanente Center for Health Research, Portland, OR 97227, USA
- 3University of Pennsylvania Perelman School of Medicine, Philadelphia, PA 19104, USA
- 4Colorado Health Outcomes Program, University of Colorado School of Medicine, Aurora, CO 80045, USA
- 5University of California Irvine School of Medicine, Irvine, CA 92697, USA
- 6Clinical Research Support, Fred Hutchinson Cancer Research Center, Seattle WA 98109, USA
- 7Internal Medicine, University of Iowa, Iowa City, IA 52242, USA
- 8Clinical Service Group, Hospital Corporation of America, Nashville, TN 374022, USA
- 9HealthPartners Institute for Education and Research, Bloomington, MN 55425, USA
- 10National Cancer Institute, Bethesda, MD 20892, USA
- Correspondence to: K E Johnson
- Accepted 2 September 2014
Demonstration pragmatic clinical trials supported by the Health Care Systems Research Collaboratory of the US National Institutes of Health highlight the following lessons about building strong research partnerships between health researchers and healthcare systems:
• Participating in pragmatic clinical trials can provide healthcare systems with evidence and tools to improve healthcare, and researchers with opportunities to conduct high impact clinical studies
• Pragmatic clinical trials answer questions relevant to healthcare systems, so clinicians, healthcare managers, health information technology (IT) staff, and clinical operations staff need to be involved in the study design
• A successful pragmatic clinical trial starts with a strong partnership between researcher and healthcare system, goes through a rigorous objective evaluation of the ability of the partner healthcare system(s) to participate, and ends with evidence about sustainable ways to improve care, as well as a long term scientific relationship
Pragmatic clinical trials are comparative effectiveness studies conducted in real-world settings to answer questions relevant to patients, clinicians, and healthcare decision makers. In contrast, explanatory clinical trials study how treatments or interventions work in carefully controlled settings and study populations, often to investigate a biological hypothesis or test a drug or device to meet regulatory requirements. The pragmatic approach requires unique methods to achieve the goal of identifying sustainable, generalizable, evidence based ways to improve healthcare.1 2 3 4 5 While previous authors addressed statistical considerations,6 little guidance is available on the methods for establishing partnerships between researchers and the people and processes in usual care settings that are necessary to conduct a pragmatic clinical trial. Based on insights from an initiative to accelerate pragmatic research,7 this article summarizes best practices for researchers and partners in healthcare systems as they establish collaborative relationships, develop research questions, and implement sustainable pragmatic clinical trials.
Pragmatic trial example
A recent example of pragmatic research is the REDUCE MRSA trial by Huang et al, in partnership with Harvard Pilgrim Health Care (analytical center), Rush University (microbiology core), and Hospital Corporation of America (healthcare system).8 Previous studies identified screening, contact precautions, targeted decolonization, and universal decolonization of patients in intensive care units as candidate strategies to prevent healthcare associated infections, particularly by meticillin resistant Staphylococcus aureus (MRSA). However, the most effective usual care strategy was not known. The REDUCE MRSA project randomized more than 40 hospitals to address this practical issue and found that treating the entire intensive care population with daily antiseptic baths and a nasal topical antibiotic was more effective than either targeted decolonization or screening and contact precautions without decolonization.
The project developed because of shared interests among hospital management, the hospital’s infection prevention and control teams, and academic researchers. Because it involved changes to care processes, launching the trial required strong support from senior hospital management, as well as input from staff at each study site who were already designated as quality improvement champions. These champions used their knowledge and experience in implementing hospital quality improvement campaigns to introduce the study protocol and training materials in a way that was familiar and acceptable to frontline clinical staff. No researchers were on site: The study depended on usual quality improvement mechanisms with the aid of twice monthly coaching calls to train and support the local champions, introduce the training materials, and coordinate assessment of protocol compliance.8
Pragmatic trial design and methodological features
A defining characteristic of pragmatic trials is that they are designed to determine what interventions work for patients receiving typical care.9 The supplemental table provides additional examples of pragmatic trials and their implications for clinical practice. Clinical trials are seldom purely explanatory or pragmatic but fall on a continuum.4 None the less, trials that are more pragmatic have particular features. Pragmatic trials are more likely than explanatory trials to study interventions already used in practice than, for example, an experimental intervention such as a new drug versus a placebo. Studies must be designed accordingly:
Rather than randomizing individual patients to different treatments, pragmatic trials may randomize clinics, hospitals, or clusters of facilities to facilitate study implementation
Pragmatic trials often use existing resources such as electronic health records and population characteristics from censuses for study design, efficient participant recruitment, intervention implementation, and data collection.4 Using electronic health records requires careful attention to missing data
As the example above illustrates, designing and implementing a pragmatic trial requires close coordination between researchers and healthcare providers and staff.3 4 5 The specific considerations and methods involved in this coordination are not currently part of the literature or the training of most researchers and practitioners. Common questions for researchers and healthcare providers considering a pragmatic clinical trial include:
What would motivate a clinic, hospital, or entire healthcare organization (which we refer to collectively in this paper as healthcare system) to participate in a research study, given the existing demands on clinical staff?
How can a pragmatic intervention be designed to fit the workflow within which it will be administered?
What features do the intervention and trial need to ensure sustainable, translatable results?
Answering these questions requires planned interactions among researchers, healthcare system managers, and frontline providers, who usually work in different environments. As described by one of the authors (PJM), “research is highly structured, like classical music, while the nature of clinical practice is to be flexible, like improvisational jazz. Pragmatic clinical trials use rigorous scientific methods to answer research questions while making adjustments to create a study protocol that clinical settings are willing and able to conduct.”
Advancing pragmatic trial partnerships
In 2006, the US National Institutes of Health launched the Health Care Systems Research Collaboratory, an initiative to promote collaborative research with healthcare organizations.7 The Collaboratory funded seven demonstration projects in which researchers are partnering with healthcare organizations to conduct pragmatic clinical trials to answer important healthcare questions. To our knowledge, this is the first pragmatic trial program designed to address both clinical questions and methods for effectively designing and implementing research in everyday healthcare settings.
All trials had a pilot period. Structured workgroups provide investigators with opportunities to share their experiences in pragmatic trial design. Discussion topics include statistics, ethics, and interactions with healthcare organizations. In 2013, we (CT, ET, and EBL) conducted 30-minute semi-structured telephone interviews with the research leaders of the demonstration projects (table 1⇓), three representatives from participating healthcare organizations, and four experts on collaborative medical research. We asked them what motivates clinical organizations to participate in research, what barriers they encountered in establishing pragmatic trials, and how they overcame them.
Based on the responses, we developed research partnership guidelines summarized in the framework in the figure⇓. The framework shows how researchers, delivery system representatives, and, increasingly, patients work together to design a pragmatic trial in several steps described in this paper: (1) build partnerships, (2) define clinically important questions, (3) assess feasibility, (4) involve stakeholders in study design, and (5) develop study workflows. For each step, we address limitations and provide guidance.
Several of the demonstration trial partnerships began with prior studies, ranging from feasibility studies to full trials that needed to be translated to new settings. In the US, researchers and healthcare organization collaborators connect through varied mechanisms including consortia such as the HMO Research Network or the Clinical and Translational Science Awards Consortium supported by the National Institutes of Health. Other countries and regions may have similar structures for fostering collaboration, such as the Academic Health Science Networks or the Collaborations for Leadership in Applied Health Research and Care in the UK.
A challenge to collaboration is that researchers and healthcare system representatives must understand and respect each others’ needs, objectives, and work culture. Interviewees said that the healthcare organization should have a culture that values research, measurement, and evidence, while researchers must understand healthcare system priorities and processes.12 For example, in the REDUCE MRSA trial, the collaborators had shared interests in preventing healthcare associated infections but needed to develop mechanisms to work together to design an effective trial. These mechanisms included Hospital Corporation of America developing a team for research and academic affairs to work with other external partners. This team meets regularly to evaluate progress on current research but also to evaluate and design new studies of mutual interest. This relationship then fostered additional studies such as the ABATE Infection trial (table 1⇑). Collaborators who are “boundary spanners” with both research and clinical experience are valuable because they facilitate the communication between researchers and clinical partners that is necessary for conducting a pragmatic trial and translating its results to usual care.13 14 The next step is defining the research questions that drive the study.
Select the research topic
Pragmatic trials answer questions that are relevant to patients, providers, and healthcare organizations.2 Questions might originate from academia, delivery systems, professional organizations, patient-clinician alliances, the public health community, or the general public. The National Institute for Health Research and the James Lind Alliance in the UK both offer mechanisms for prioritizing research that is most wanted by patients. The research topic for several of the Health Care Systems Collaboratory demonstration projects arose from discussions with healthcare system managers. For example, the PPACT study (table 1⇑) emerged from what seemed like an intractable problem to patients, clinicians, and researchers about how to control chronic pain.
To be feasible, pragmatic trials must potentially add value to the healthcare system. Without this element, researchers are unlikely to receive the necessary clinical and operational commitment for the study. Therefore, research topics should fit a healthcare system’s need for (1) results that decision makers might use to improve care and patient outcomes, (2) integration with clinic workflow, (3) relatively low entry costs, and (4) positive, or at least little negative, impact on productivity and the healthcare system’s finances. Not all trials will meet all criteria. A limitation is that some trials must substantially disrupt clinical workflow; however, these trials can be feasible with effective joint planning and a potentially large clinical payoff. For example, in the REDUCE MRSA trial, the practical experience of the academic investigators as medical directors of infection prevention programs and the health system quality improvement experience of the Hospital Corporation of America co-investigators enabled the creation of functional and efficient changes to workflow to implement either targeted or universal decolonization that were not only acceptable by management and clinicians but were used for enduring implementation and dissemination after the trial results were known. Pragmatic trials must watch for conflicts with delivery system quality improvement projects. For example, for a demonstration trial embedded in a delivery system that was transitioning to a new electronic health record, researchers and delivery system representatives cooperated to limit changes in the electronic health record that might affect the study.
Pilot phases are particularly important to understand the best study process, resources, and management to move work into everyday workflow.15 All of the Collaboratory demonstration projects were funded as pilots in order to assess feasibility of full implementation. Even for previously successful research partnerships, an objective pre-assessment of the healthcare system’s capabilities for the proposed study is essential. Critical questions include:
Are sufficient patient numbers and data available for the analysis?
Can data be collected at all clinical sites?
How do the sites vary in services and capabilities?
Can the system’s regulatory and administrative infrastructure support approval and oversight by ethics committees and review boards?
Will the intervention add long-term value to the system?
If the pre-assessment process reveals a poor fit between a healthcare system and a planned study, the system cannot participate, although researchers can maintain relationships with its management for future studies and dissemination.
Involve stakeholders in design
A successful pragmatic clinical trial requires engagement and input from all levels of a healthcare system. Pragmatic trials give clinical staff a professional development opportunity to practice research skills and participate in quality improvements.16 17 However, clear communication between researchers and clinical personnel about processes, expectations, and roles and goals is essential (table 2⇓), since the collaboration is often a new experience for all parties. All collaborators should acknowledge competing priorities and differences in work culture.18 For example, researchers may discount the ongoing quality improvement work in healthcare systems as not rigorous, while frontline clinicians may perceive research as slow, expensive, inflexible, or not clinically relevant. As described earlier, the REDUCE MRSA trial developed a successful trial by working closely with staff who were already designated as quality improvement champions about how to implement and monitor the protocol. Weekly steering committee and analytic calls were used to maintain communication across multiple stakeholders.
Getting the attention of busy managers is challenging, especially before funding is assured. Researchers approaching healthcare organization managers to propose research embedded in clinical practice should highlight advantages such as the potential for gains in patient outcomes, staff efficiency, or health information technology (IT) improvements, along with congruence with other organization-wide priorities. After getting leadership buy-in, interviewees recommended networking to find people throughout the organization with the knowledge, interest, and authority to contribute to the study, as well as the time to maintain regular contact with researchers.
Researchers we interviewed stated that clinical staff, healthcare system managers, and researchers need to work together to optimize design and implementation of the study protocol. This is especially important during design and piloting. Staff and managers know how to best use existing health IT, workflow, clinical procedures, and local champions to make study participation easier for clinical staff. For example, based on provider input, one demonstration project moved data collection from a study website to an existing clinical system, another changed an intervention from group to individual therapy, and a third shifted implementation responsibilities from doctors to pharmacists.
Research teams may also need to involve clinical managers and frontline staff to identify additional costs of the study. These might include wear and tear on medical devices, use of additional supplies and services, time and effort to collect additional data, or the need for additional staff or providers. In the UK, for example, clear demarcation and calculation of which costs are the responsibility of the researcher and which are the responsibility of the healthcare service are standard parts of grant applications to national funders. In countries with multiple payers, operational staff can prevent double billing and advise research teams about sustainability, such as who will pay for intervention costs when the trial ends.
Health IT staff are increasingly crucial to pragmatic trials because of the growing use of electronic health records in research. In our experience, many studies experienced challenges making additions or alterations to the electronic health record. These were resolved either by consulting with clinical IT staff or by changing study plans.
Implement the study into workflow
To facilitate study implementation by clinical staff, trial procedures should mimic normal clinical practices and use existing resources as much as possible. Because pragmatic clinical trials occur in the context of dynamic real-world care, researchers need to be prepared to adapt studies based on stakeholder input or changes in the care environment. Ideally this happens during a pilot phase but likely will continue even as the study is under way. Flexibility in adapting to specific clinical situations will improve study protocol compliance by clinicians and facilitate the long term sustainability of tested interventions.
Planning for sustainability from the beginning of a study lays a foundation for implementing and spreading successful changes, including establishing a sustainable funding model.14 A limitation of research embedded in practice is that researchers must consider how to remove or adapt an intervention if data indicate that it is not effective. This part of the study is facilitated by communicating tailored results to all groups involved in the study, including patients and their families.
Pragmatic medical research is receiving increasing attention and funding.12 19 20 Pragmatic trials are conducted in clinical settings for “a real-world test in a real-world population.”11 As our examples from the REDUCE MRSA trial and the NIH Health Care Systems Research Collaboratory illustrate, pragmatic trials require that researchers and healthcare system clinicians, senior management, and staff develop the attitudes, skills, resources, and shared vision for close collaboration. The insights presented above, together with the resources in the Linked information box, provide guidance for building engaged pragmatic clinical trial teams.
NIH Collaboratory. Health Care Systems Research Collaboratory. www.nihcollaboratory.org/Pages/Knowledge-Repository.aspx
NIH Collaboratory. Introduction to pragmatic clinical trials. http://sites.duke.edu/rethinkingclinicaltrials/introduction-to-pragmatic-clinical-trials/
Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Clin Epidemiol 2009;62:499-505.
NIH Collaboratory. Rethinking clinical trials: a living textbook of pragmatic clinical trials. http://sites.duke.edu/rethinkingclinicaltrials/
CTSA Consortium. CTSA: Clinical & Translational Science Awards. https://www.ctsacentral.org/
HMO Research Network Research Tools. www.hmoresearchnetwork.org/en/Tools%20&%20Materials/
Research Toolkit: A toolkit for health research in partnership with practices and communities. http://researchtoolkit.org/
James Lind Alliance. www.lindalliance.org/. Discusses stakeholder involvement in research priority setting.
NIHR: National Institute for Health Research. Identifying research questions. www.nets.nihr.ac.uk/identifying-research
NHS England. Academic health science networks. www.england.nhs.uk/ourwork/part-rel/ahsn/
NIHR: National Institute for Health Research. Collaborations for leadership in applied health research and care. www.nihr.ac.uk/about/collaborations-for-leadership-in-applied-health-research-and-care.htm
Cite this as: BMJ 2014;349:g6826
Contributors: All authors contributed to the content, drafting, or revision of this article and approved the final version. KEJ and CT drafted the manuscript and model. CT and ET led data collection. Other authors provided input on lessons learnt from their pragmatic experience. EBL is the guarantor.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: this work was supported by a cooperative agreement (U54 AT007748) from the NIH Common Fund for the NIH Health Care Systems Research Collaboratory; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; SSH received non-financial support from Sage Products and Molnlycke.