Standards for Reporting Implementation Studies (StaRI) StatementBMJ 2017; 356 doi: https://doi.org/10.1136/bmj.i6795 (Published 06 March 2017) Cite this as: BMJ 2017;356:i6795
- Hilary Pinnock, professor of primary care respiratory medicine1,
- Melanie Barwick, senior scientist, head of Child and Youth Mental Health Research Unit2,
- Christopher R Carpenter, associate professor, emergency medicine3,
- Sandra Eldridge, professor of biostatistics4,
- Gonzalo Grandes, head of unit, coordinator of primary health care research, Biocruces Research Institute5,
- Chris J Griffiths, professor of primary care6,
- Jo Rycroft-Malone, pro-vice chancellor, research and impact7,
- Paul Meissner, director of research program development8,
- Elizabeth Murray, professor of ehealth and primary care, and head of research9,
- Anita Patel, professor of health economics6,
- Aziz Sheikh, director1,
- Stephanie J C Taylor, professor in public health and primary care6
- for the StaRI Group
- 1Asthma UK Centre for Applied Research, Usher Institute of Population Health Sciences and Informatics, University of Edinburgh, Edinburgh EH8 9AG, UK
- 2Research Institute, Hospital for Sick Children; Department of Psychiatry and Dalla Lana School of Public Health, University of Toronto, Canada
- 3Washington University Division of Emergency Medicine. Washington University School of Medicine in St Louis, USA
- 4Pragmatic Clinical Trials Unit, Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, UK
- 5Primary Care Research Unit of Bizkaia, Basque Health Service, Spain
- 6Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, UK
- 7Bangor Institute for Health & Medical Research, Bangor University, UK
- 8Montefiore Medical Center, The University Hospital for Albert Einstein College of Medicine, New York, USA
- 9Research Department of Primary Care and Population Health, University College London, UK
- Correspondence to: H Pinnock
- Accepted 27 October 2016
Implementation studies are often poorly reported and indexed, reducing their potential to inform initiatives to improve healthcare services. The Standards for Reporting Implementation Studies (StaRI) initiative aimed to develop guidelines for transparent and accurate reporting of implementation studies. Informed by the findings of a systematic review and a consensus-building e-Delphi exercise, an international working group of implementation science experts discussed and agreed the StaRI Checklist comprising 27 items. It prompts researchers to describe both the implementation strategy (techniques used to promote implementation of an underused evidence-based intervention) and the effectiveness of the intervention that was being implemented. An accompanying Explanation and Elaboration document (published in BMJ Open, doi:10.1136/bmjopen-2016-013318) details each of the items, explains the rationale, and provides examples of good reporting practice. Adoption of StaRI will improve the reporting of implementation studies, potentially facilitating translation of research into practice and improving the health of individuals and populations.
Underpinning the 27 item StaRI Checklist is the concept of dual strands describing (a) the strategies used to promote implementation and (b) the intervention being implemented
The expectation is that the authors will clarify both (a) how they anticipate that the strategies employed are likely to promote implementation and (b) explain the underpinning premise of why implementation of the intervention may be expected to improve healthcare or health outcomes
Unlike most reporting standards that apply to a specific research methodology, the StaRI Statement and accompanying Checklist refers to the broad range of study designs employed in implementation science
The requirement for extensive description of context, implementation strategies, and interventions, as well as reporting a broad range of effectiveness, process, and health economic outcomes, will challenge journals operating strict word limits for research papers and may require (innovative) solutions and use of supplementary online materials
Globally, healthcare systems are struggling to deliver the benefits of research to their populations.1 w1-w3 Increasingly, it is recognised that translation from “bench to bedside to community”w4 is often ineffective and inefficient. The scientific community needs to focus on how effective interventions are disseminated and implemented across the spectrum of contexts and settings in order to improve individual and population health.w5 Against this background, implementation science has emerged as an important discipline for developing the evidence base on how to translate research findings into routine care.1 2 3 4
Implementation studies are, however, often poorly reported and indexed, making it difficult to find, reproduce, or synthesise the evidence from relevant studies.5 More specific criticisms include poor (or absent) descriptions of conceptual frameworks underpinning the research,5 6 inadequate description of context, and incomplete information about how the intervention was promoted and implemented (or not) in the different settings.6 w6 w7 Similar concerns with, for example, the reporting of randomised controlled trials (RCTs) led to the introduction of the Consolidated Standards of Reporting Trials (CONSORT) checklist,w8 with evidence of subsequent improvement in reporting standards.w9-w11 There have been calls for the development of similar standards for transparent and accurate reporting of implementation studies.5 6 7 The Standards for Reporting Implementation Studies (StaRI) initiative aimed to address this need.
Scope and relationship with other reporting standards
Implementation science encompasses a broad range of methodologies applicable to improving the dissemination, implementation, and scaling up of effective behavioural, clinical, healthcare, public health, global health, and educational interventions8 (or discontinuation of ineffective or harmful practicesw12) with a view to improving quality of care and health outcomes. Although this document is set within the context of healthcare and population health, there are parallels in other domains (such as educational initiatives).9 w13 The StaRI Statement and Checklist may thus have resonance outside healthcare.
Understanding of the position that implementation studies hold in the science of developing, evaluating, disseminating, and implementing healthcare interventions has been evolving over recent years. The UK Medical Research Council (MRC) Framework for Development and Evaluation of Complex Interventions10 emphasises the need to disseminate and implement findings of complex interventions trials but offers no advice on how to achieve this. PRECIS-2 conceives trial design on an “explanatory-pragmatic” spectrum11 but does not project beyond pragmatic trials to implementation in routine practice. Neither of these frameworks addresses the need for research to explore how interventions shown to be effective in trials require adaptation if they are to align with the routines of practice and be successfully implemented into “usual care” settings.12 w14-w17 Implementation science undertakes studies that explore healthcare contexts, develop and evaluate strategies for implementing effective interventions that address local realities, can be implemented at scale and are potentially sustainable.2 w18 Proposed frameworks have added an “implementation cycle” to complement the MRC’s complex intervention cycle,13 or extended a linear spectrum (see fig 1⇓).7 14 Others have emphasised the potential overlap between effectiveness and implementation research and described “hybrid” designs.15
The StaRI Statement and Checklist are intended to improve reporting of implementation studies, employing a range of study designs to develop and evaluate implementation strategies with the aim of enhancing adoption and sustainability of effective interventions.16 Implementation studies may be distinguished from quality improvement reports that describe system level initiatives, typically in the context of a specific problem within a specific healthcare system,16 w19 and the World Health Organisation guidelines, which focus on improving reporting of their fieldwork.w20
We followed the methodology described in the Developing Health Research Reporting Guidelines.w21 Our full protocol is available on the EQUATOR website.w22 After a systematic literature review, we recruited international multidisciplinary experts (including healthcare researchers, journal editors, healthcare professionals and managers, methodologists, guideline developers, patient organisations, and funding bodies) to participate in an e-Delphi exercise.17 Of 66 experts approached, 23 contributed suggestions for the Checklist, 20 completed the first scoring round, and 19 completed the second scoring round. Of 47 potential items, 35 reached the a priori level of consensus for inclusion—that is, 80% agreement with priority scores 7, 8, or 9—and 19 items achieved 100% agreement. All these items, with their final priority scores, were taken forward as candidate items for inclusion in the StaRI Checklist.
In April 2015, we convened a two-day consensus working group in London attended by 15 international delegates (UK or Europe=11, US or Canada=4) drawn from multiple disciplines. Delegates included healthcare researchers (n=9), journal editors (n=6), healthcare professionals (n=8) and managers (n=1), methodologists (n=4), guideline developers (n=2), and funding bodies (n=2) (several participants had more than one role). This group discussed the candidate items and agreed the first draft of the StaRI Checklist. The discussions were informed by the outcome of the e-Delphi exercise (see appendix 1 for the e-Delphi results as provided to the workshop delegates), but items were also considered in the context of other published reporting standards and the wider literature, and the working group’s expertise in implementation science. After general discussion on key defining concepts (informed by points raised in the e-Delphi17), each candidate item was considered in turn. Agreement was reached by discussion rather than by consensus scoring. The initial draft statement and documents were subsequently developed iteratively by email discussion.
Constructive feedback on a penultimate draft of the StaRI Statement from colleagues working internationally in the field of implementation science, healthcare researchers, clinicians, and patients was used to help shape the final version of the paper. In addition, we presented the concepts to and sought feedback from several workshops, conference discussions, and implementation project steering groups.
There are two defining concepts underpinning the StaRI Statement and Checklist. The first is the dual strands of describing, on the one hand, the implementation strategy and, on the other, the clinical, healthcare, global health, or public health intervention being implemented.3 w23 For example, an implementation strategy (staff training, changes to invitation letters and appointment systems, development of computer templates, ongoing audit, etc) might support an intervention (such as offering the option of telephone consultations) with the aim of improving access to routine asthma care.w24 These strands are represented as two columns in the Checklist (see table 1⇓). The primary focus of implementation science is the implementation strategy,w25 and the expectation is that the items in column 1 will always be fully completed with details of how the intervention was implemented and the impact measured as an implementation outcome. The second strand (column 2) expects authors to complete items about the impact of the intervention on the health of the target population. This may be measured as a health outcome, or it may be more appropriate to cite robust evidence to support known beneficial effects of the intervention on health of individuals or populations (such as reducing smoking prevalence). Even when evidence is strong, the possibility that the impact of an intervention may be attenuated when it is implemented in routine practice needs to be considered. Although all items are worthy of consideration, not all items will be applicable to, or feasible in, every study; a fully completed StaRI Checklist may thus include a number of “not applicable” items.
The second concept is that, unlike most reporting standards that apply to a specific research methodology, StaRI applies to the broad range of research methodologies employed in implementation science (for example, cluster RCTs, controlled clinical trials, interrupted time series, cohort, case study, before and after studies, as well as mixed methods for quantitative or qualitative assessments).3 Authors are referred to other reporting standards for advice on reporting specific methodological features—for example, randomisation in cluster RCTs,w26 matching criteria in cohort studies,w27 or addressing reflexivity in qualitative research.w28
The StaRI Checklist
The StaRI Checklist comprises 27 items, of which 10 items expect authors to consider the dual strands of the implementation strategy and the intervention (see table 1⇑). Details about each of the Checklist items is provided in the accompanying Explanation and Elaboration document published in BMJ Open.18 Appendix 2 is a version of the Checklist for completion by authors submitting an implementation paper. It is strongly recommended that authors using the StaRI Checklist read the detailed document that explains the rationale for each item and provides examples of good practice.
Three overarching components are emphasised in the Checklist:
The expectation is that authors have an explicit hypothesis (we use the term “logic pathway”) that spans both how the implementation strategy is expected to work and the mechanism by which the intervention is expected to improve healthcare (see Explanation and Elaboration document18 for a table of alternative terminologies and a link to a detailed description of “logic models”). This logic pathway should reflect the rationale presented in the introduction, determine the approach to implementation, dictate implementation, health, and process outcomes, and provide insights into why and how the implementation strategy and intervention worked (or not).
The balance between fidelity to, and adaptation of, the implementation strategy and intervention is of particular interest in implementation science. Fidelity refers to the degree of adherence to the described implementation strategy and intervention; adaptation is the degree to which users modify the strategy and intervention during implementation to suit the local needs (see table 2⇓ for further description and examples). Insufficient fidelity to the “active ingredients” of an intervention may dilute effectiveness,w29 whereas insufficient adaptation or tailoring to local context may inhibit effective implementation.19 An approach to reporting these apparently contradictory concepts is to define the core components of an intervention (ideally related to the logic pathway) to which fidelity is expected, and those aspects that may be adapted by local sites to aid implementation.19 w30
Successful implementation of an intervention into practice is a planned, facilitated process involving the interplay between individuals, intervention or new ways of working, and context to promote evidence-informed practice.20 A rich description of the context is critical to enable the reader to assess the external validity of the reported study,w18 and to decide how the context in the study compares to their situation and whether the implementation strategy can be directly transposed or will need adapting.w31 Similarly, social, political, and economic context influence the “entrenched practices” that hinder evidence-based de-implementation of unproven practices or interventions.w12 w32
Implementation science is an emerging and rapidly evolving field. The StaRI Statement and Checklist should therefore be seen as an evolving document, and potentially a catalyst for discussing and defining how implementation studies are conceived, planned, and reported.5
We hope that the concept of dual strands will resonate with researchers designing and reporting implementation science studies. We appreciate that the distinction will not always be as unambiguous as it seems in the StaRI checklist, but we suggest that considering the design and evaluation of implementation studies in these two strands is helpful and aids clarity of study design and reporting. We also recognise that not all studies will measure health outcomes, though consideration of the ultimate goal of improving health through implementing an evidence-based intervention would seem a reasonable requirement. Feedback on this underpinning concept will be valuable for future iterations of the StaRI Statement and Checklist.
There are two practical challenges for the application of StaRI that warrant discussion. First, implementation science uses diverse methodologies that need to be accommodated in the reporting standards. One option is to incorporate relevant items from other checklists, but this may be perceived as limiting the methodological options. StaRI therefore advises authors to consult methodological checklists for reporting design-specific aspects of their chosen study design. By doing this, we have implicitly prioritised the concept underpinning implementation studies, though this should not be interpreted as undermining the rigour of reporting the chosen study design.
The second challenge is the requirement for extensive description of context, implementation strategies, and interventions as well as reporting a broad range of primary effectiveness, process, health, economic, and implementation outcomes.w25 This requirement will stimulate debate about word counts, supplementary material, and additional publications in order to accommodate journal requirements, author needs, and reader preferences. This tension is further discussed in the Explanation and Elaboration document,18 and some practical approaches are suggested for summarising information in tables or figures. We look forward to learning how authors and journals work with these challenges and the (innovative) solutions that they adopt (such as appendices, supplementary online files, and additional publications).
The StaRI Statement is registered with the EQUATOR Network (www.equator-network.org), and the Checklist (for completion by authors) is freely available from bmj.com (appendix 2). We invite editors of all journals publishing implementation research to consider requiring submission of a StaRI Checklist, and authors reporting their implementation studies to use the Checklist. In the future we would like to work with authors as they apply the Checklist to their papers, “road testing” the standards and enabling iterative development.
Previously published reporting guidelines have been instrumental in improving reporting standards,6 w8-w10 and our hope is that StaRI will achieve a similar improvement in the reporting of implementation strategies that will facilitate translation of effective interventions into routine practice, ultimately to benefit the health of individuals and populations.
Enhancing the QUAlity and Transparency Of health Research (EQUATOR). www.equator-network.org/
Consolidated Framework For Implementation Research (CFIR). www.cfirguide.org/imp.html
Dissemination and implementation models in Health Research and Practice. www.dissemination-implementation.org/index.aspx
Process evaluation of complex interventions. www.ioe.ac.uk/MRC_PHSRN_Process_evaluation_guidance_final(2).pdf
Reach Effectiveness Adoption Implementation Maintenance (RE-AIM). www.re-aim.hnfe.vt.edu
Members of the the StaRI Group are: Melanie Barwick, Chris Carpenter, Peter Craig, Sandra Eldridge, Eleni Epiphaniou, Gonzalo Grandes Odriozola, Chris Griffiths, Martin Gulliford, Jo Rycroft-Malone, Paul Meissner, Brian Mittman, Elizabeth Murray, Anita Patel, Gemma Pearce, Hilary Pinnock, Aziz Sheikh, and Steph Taylor.
Members of the PRISMS team (Eleni Epiphaniou, Gemma Pearce, and Hannah Parke) supported the underpinning literature work, and the e-Delphi exercise was handled by ClinVivo. We thank colleagues (implementation science experts, healthcare researchers, clinicians, PhD students) who reviewed the penultimate draft of the StaRI statement and provided a reality check and constructive feedback: Helen Ashdown, David Chambers, Louise Craig, Clarisse Dibao-Dina, Peter Hanlon, Roger Jones, Rachel Jordan, Chris del Mar, Brian McKinstry, Susan Morrow, John Ovretveit, David Price, Kamran Siddiqui, Rafael Stelmach, Paul Stephenson, Shaun Treweek, Bryan Weiner. We also thank Melissa Goodbourn and Allison Worth, who arranged feedback from the Edinburgh Clinical Research Facility Patient Advisory Panel (Stephanie Ashby, Alison Williams), and Steven Towndrow who coordinated feedback from the Patient and Public Involvement representatives of the NIHR CLAHRC North Thames (Ben Wills-Eve, Rahila Bashir, Julian Ashton, Colleen Ewart, Karen Williams).
Contributors: HP initiated the idea for the study and with ST led the development of the protocol, securing of funding, study administration, workshop, and writing of the paper. AS, CG, and SE advised on the development of the protocol and data analysis. All authors participated in the StaRI international working group along with GP, BM, MG. HP wrote the initial draft of the paper, to which all the authors contributed. HP is the study guarantor.
Funding: The StaRI initiative and workshop were funded by contributions from the Asthma UK Centre for Applied Research (AC-2012-01); Chief Scientist Office, Scottish Government Health and Social Care Directorates (PCRCA_08_01); the Centre for Primary Care and Public Health, Queen Mary University of London; and with contributions in kind from the PRISMS team (NIHR HS&DR Grant 11/1014/04). ST was (in part) supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Bart’s Health NHS Trust. AS is supported by the Farr Institute. The funding bodies had no role in the design, in the collection, analysis, and interpretation of data; in the writing of the manuscript; nor in the decision to submit the manuscript for publication.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: research grants from Chief Scientist Office (HP), Asthma UK (AS, HP, ST), Farr Institute (AS), NIHR HS&DR (HP, ST), NIHR CLAHRC (ST) for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; CC is deputy editor-in-chief for Academic Emergency Medicine and on the editorial boards for the Journal of the American Geriatrics Society and Annals of Internal Medicine's ACP Journal Club and serves as paid faculty for Emergency Medical Abstracts, JR-M is director of the NIHR HS&DR Programme, no other relationships or activities that could appear to have influenced the submitted work.
Disclaimers: The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health.
Provenance of the paper: The StaRI Checklist was informed by the findings of a literature review and an e-Delphi exercise, an international consensus workshop, and the subsequent email correspondence among members of the StaRI Group. The international authors contributed expertise on clinical practice, public health, knowledge exchange, implementation science, complex interventions, and a range of methodologies including quantitative and qualitative evaluations.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.