BMJ 1998;317:465-468 ( 15 August )

Education and debate

Getting research findings into practice

Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings

This is the seventh in a series of eight articles analysing the gap between research and practice

Lisa A Bero, associate professor, a Roberto Grilli, head, b Jeremy M Grimshaw, programme director, c Emma Harvey, research fellow, d Andrew D Oxman, director, e Mary Ann Thomson, senior research fellow, c on behalf of the Cochrane Effective Practice and Organisation of Care Review Group.

a Institute for Health Policy Studies, University of California at San Francisco, 1388 Sutter Street, 11th floor, San Francisco, CA 94109, USA, b Unit of Clinical Policy Analysis, Laboratory of Clinical Epidemiology, Istituto di Ricerche Farmacologiche Mario Negri, Via Eritrea 62, 20157 Milan, Italy, c Health Services Research Unit, Department of Public Health, Aberdeen AB25 2ZD, d Department of Health Sciences and Clinical Evaluation, University of York, York YO1 5DD, e Health Services Research Unit, National Institute of Public Health, PO Box 4404 Torshov, N-0462 Oslo, Norway

Correspondence to: Dr Grimshaw j.m.grimshaw{at}abdn.ac.uk

Series editors: Andrew Haines and Anna Donald

Despite the considerable amount of money spent on clinical research relatively little attention has been paid to ensuring that the findings of research are implemented in routine clinical practice.1 There are many different types of intervention that can be used to promote behavioural change among healthcare professionals and the implementation of research findings. Disentangling the effects of intervention from the influence of contextual factors is difficult when interpreting the results of individual trials of behavioural change.2 Nevertheless, systematic reviews of rigorous studies provide the best evidence of the effectiveness of different strategies for promoting behavioural change. 3 4 In this paper we examine systematic reviews of different strategies for the dissemination and implementation of research findings to identify evidence of the effectiveness of different strategies and to assess the quality of the systematic reviews.

Summary points

Systematic reviews of rigorous studies provide the best evidence on the effectiveness of different strategies to promote the implementation of research findings
Passive dissemination of information is generally ineffective
It seems necessary to use specific strategies to encourage implementation of research based recommendations and to ensure changes in practice
Further research on the relative effectiveness and efficiency of different strategies is required

    Identification and inclusion of systematic reviews

We searched Medline records dating from 1966 to June 1995 using a strategy developed in collaboration with the NHS Centre for Reviews and Dissemination. The search identified 1139 references. No reviews from the Cochrane Effective Practice and Organisation of Care Review Group4 had been published during this time. In addition, we searched the Database of Abstracts of Research Effectiveness (DARE) (www.york.ac.uk/inst/crd) but did not identify any other review meeting the inclusion criteria.

We searched for any review of interventions to improve professional performance that reported explicit selection criteria and in which the main outcomes considered were changes in performance or outcome. Reviews that did not report explicit selection criteria, systematic reviews focusing on the methodological quality of published studies, published bibliographies, bibliographic databases, and registers of projects on dissemination activities were excluded from our review. If systematic reviews had been updated we considered only the most recently published review. For example, the Effective Health Care bulletin on implementing clinical guidelines superseded the earlier review by Grimshaw and Russell. 5 6

Two reviewers independently assessed the quality of the reviews and extracted data on the focus, inclusion criteria, main results, and conclusions of each review. A previously validated checklist (including nine criteria scored as done, partially done, or not done) was used to assess quality. 7 8 Reviews also gave a summary score (out of seven) based on the scientific quality of the review. Major disagreements between reviewers were resolved by discussion and consensus.

    Results and assessment of systematic reviews

We identified 18 reviews that met the inclusion criteria. They were categorised as focusing on broad strategies (such as the dissemination and implementation of guidelines 5 6 9-11 ), continuing medical education, 12 13 particular strategies (such as audit and feedback, 14 15 computerised decision support systems, 16 17 or multifaceted interventions18), particular target groups (for example, nurses19 or primary healthcare professionals20), and particular problem areas or types of behaviour (for example, diagnostic testing,15 prescribing,21 or aspects of preventive care 15 22-25 ). Most primary studies were included in more than one review, and some reviewers published more than one review. No systematic reviews published before 1988 were identified. None of the reviews explicitly addressed the cost effectiveness of different strategies for effecting changes in behaviour.

There was a lack of a common approach adopted between the reviews in how interventions and potentially confounding factors were categorised. The inclusion criteria and methods used in these reviews varied considerably. Interventions were frequently classed differently in the different systematic reviews.

Common methodological problems included the failure to adequately report criteria for selecting studies included in the review, the failure to avoid bias in the selection of studies, the failure to adequately report criteria used to assess validity, and the failure to apply criteria to assess the validity of the selected studies. Overall, 42% (68/162) of criteria were reported as having been done, 49% (80/162) as having been partially done, and 9% (14/162) as not having been done. The mean summary score was 4.13 (range 2 to 6, median 3.75, mode 3).

Encouragingly, reviews published more recently seemed to be of better quality. For studies published between 1988 and 1991 (n=6) only 20% (11/54) of criteria were scored as having been done (mean summary score 3.0); for reviews published after 1991 (n=12) 52% (56/108) of criteria were scored as having been done (mean summary score 4.7).

Five reviews attempted formal meta-analyses of the results of the studies identified. 12 17 19 23 25 The appropriateness of meta-analysis in three of these reviews is uncertain, 12 17 19 and the reviews should be considered exploratory at best, given the broad focus and heterogeneity of the studies included in the reviews with respect to the types of interventions, targeted behaviours, contextual factors, and other research factors.2

A number of consistent themes were identified by the systematic reviews (box). (Further details about the systematic reviews are available on the BMJ's website.) Most of the reviews identified modest improvements in performance after interventions. However, the passive dissemination of information was generally ineffective in altering practices no matter how important the issue or how valid the assessment methods. 5 9 11 13 21 26 The use of computerised decision support systems has led to improvements in the performance of doctors in terms of decisions on drug dosage, the provision of preventive care, and the general clinical management of patients, but not in diagnosis.16 Educational outreach visits have resulted in improvements in prescribing decisions in North America. 5 13 Patient mediated interventions also seem to improve the provision of preventive care in North America (where baseline performance is often very low).13 Multifaceted interventions (that is, a combination of methods that includes two or more interventions such as participation in audit and a local consensus process) seem to be more effective than single interventions. 13 18 There is insufficient evidence to assess the effectiveness of some interventions---for example the identification and recruitment of local opinion leaders (practitioners nominated by their colleagues as influential).5

Interventions to promote behavioural change among health professionals
Consistently effective interventions

  • Educational outreach visits (for prescribing in North America)
  • Reminders (manual or computerised)
  • Multifaceted interventions (a combination that includes two or more of the following: audit and feedback, reminders, local consensus processes, or marketing)
  • Interactive educational meetings (participation of healthcare providers in workshops that include discussion or practice)

Interventions of variable effectiveness

  • Audit and feedback (or any summary of clinical performance)
  • The use of local opinion leaders (practitioners identified by their colleagues as influential)
  • Local consensus processes (inclusion of participating practitioners in discussions to ensure that they agree that the chosen clinical problem is important and the approach to managing the problem is appropriate)
  • Patient mediated interventions (any intervention aimed at changing the performance of healthcare providers for which specific information was sought from or given to patients)

Interventions that have little or no effect

  • Educational materials (distribution of recommendations for clinical care, including clinical practice guidelines, audiovisual materials, and electronic publications)
  • Didactic educational meetings (such as lectures)

Few reviews attempted explicitly to link their findings to theories of behavioural change. The difficulties associated with linking findings and theories are illustrated in the review by Davis et al, who found that the results of their overview supported several different theories of behavioural change.13

Availability and quality of primary studies
This overview also allows the opportunity to estimate the availability and quality of primary research in the areas of dissemination and implementation. Identification of published studies on behavioural change is difficult because they are poorly indexed and scattered across generalist and specialist journals. Nevertheless, two reviews provided an indication of the extent of research in this area. Oxman et al identified 102 randomised or quasirandomised controlled trials involving 160 comparisons of interventions to improve professional practice.11 The Effective Health Care bulletin on implementing clinical guidelines identified 91 rigorous studies (including 63 randomised or quasirandomised controlled trials and 28 controlled before and after studies or time series analyses).5 Even though the studies included in these two reviews fulfilled the minimum inclusion criteria, some are methodologically flawed and have potentially major threats to their validity. Many studies randomised health professionals or groups of professionals (cluster randomisation) but analysed the results by patient, thus resulting in a possible overestimation of the significance of the observed effects (unit of analysis error).27 Given the small to moderate size of the observed effects this could lead to false conclusions about the significance of the effectiveness of interventions in both meta-analyses and qualitative analyses. Few studies attempted to undertake any form of economic analysis.

Given the importance of implementing the results of sound research and the problems of generalisability across different healthcare settings, there are relatively few studies of individual interventions to effect behavioural change. The review by Oxman et al identified studies involving 12 comparisons of educational materials, 17 of conferences, four of outreach visits, six of local opinion leaders, 10 of patient mediated interventions, 33 of audit and feedback, 53 of reminders, two of marketing, eight of local consensus processes, and 15 of multifaceted interventions.11 Few studies compared the relative effectiveness of different strategies; only 22 out of 91 studies reviewed in the Effective Health Care bulletin allowed comparisons of different strategies.5 A further limitation of the evidence about different types of interventions is that the research is often conducted by limited numbers of researchers in specific settings. The generalisability of these findings to other settings is uncertain, especially because of the marked differences in undergraduate and postgraduate education, the organisation of healthcare systems, potential systemic incentives and barriers to change, and societal values and cultures. Most of the studies reviewed were conducted in North America; only 14 of the 91 studies reviewed in the Effective Health Care bulletin had been conducted in Europe.5

    The way forward

This overview suggests that there is an increasing amount of primary and secondary research in the areas of dissemination and implementation. It is striking how little is known about the effectiveness and cost effectiveness of interventions that aim to change the practice or delivery of health care. The reviews that we examined suggest that the passive dissemination of information (for example, publication of consensus conferences in professional journals or the mailing of educational materials) is generally ineffective and, at best, results only in small changes in practice. However, these passive approaches probably represent the most common approaches adopted by researchers, professional bodies, and healthcare organisations. The use of specific strategies to implement research based recommendations seems to be necessary to ensure that practices change, and studies suggest that more intensive efforts to alter practice are generally more successful.

At a local level greater attention needs to be given to actively coordinating dissemination and implementation to ensure that research findings are implemented. The choice of intervention should be guided by the evidence on the effectiveness of dissemination and implementation strategies, the characteristics of the message,10 the recognition of external barriers to change,13 and the preparedness of the clinicians to change.28 Local policymakers with responsibility for professional education or quality assurance need to be aware of the results of implementation research, develop expertise in the principles of the management of change, and accept the need for local experimentation.

Given the paucity of evidence it is vital that dissemination and implementation activities should be rigorously evaluated whenever possible. Studies evaluating a single intervention provide little new information about the relative effectiveness and cost effectiveness of different interventions in different settings. Greater emphasis should be given to conducting studies that evaluate two or more interventions in a specific setting or help clarify the circumstances that are likely to modify the effectiveness of an intervention. Economic evaluations should be considered an integral component of research. Researchers should have greater awareness of the issues related to cluster randomisation, and should ensure that studies have adequate power and that they are analysed using appropriate methods.29

The NHS research and development programme on evaluating methods to promote the implementation of research and development is an important initiative that will contribute to our knowledge of the dissemination of information and the implementation of research findings.30 However, these research issues cut across national and cultural differences in the practice and financing of health care. Moreover, the scope of these issues is such that no one country's health services research programme can examine them in a comprehensive way. This suggests that there are potential benefits of international collaboration and cooperation in research, as long as appropriate attention is paid to cultural factors that might influence the implementation process such as the beliefs and perceptions of the public, patients, healthcare professionals, and policymakers.

The results of primary research should be systematically reviewed to identify promising implementation techniques and areas where more research is required.3 Undertaking reviews in this area is difficult because of the complexity inherent in the interventions, the variability in the methods used, and the difficulty of generalising study findings across healthcare settings. The Cochrane Effective Practices and Organisation of Care Review Group is helping to meet the need for systematic reviews of current best evidence on the effects of continuing medical education, quality assurance, and other interventions that affect professional practice. A growing number of these reviews are being published and updated in the Cochrane Database of Systematic Reviews. 4 31

The articles in this series are adapted from Getting Research Findings into Practice, edited by Andrew Haines and Anna Donald and published by BMJ Books.

    Acknowledgments

This paper is based on a briefing paper prepared by the authors for the Advisory Group on the NHS research and development programme on evaluating methods to promote the implementation of research and development. We thank Nick Freemantle for his contribution to this paper.

Funding: This work was partly funded by the European Community funded Eur-Assess project. The Cochrane Effective Practice and Organisation of Care Review Group is funded by the Chief Scientist Office of the Scottish Office Home and Health Department; the NHS Welsh Office of Research and Development; the Northern Ireland Department of Health and Social Services; the research and development offices of the Anglia and Oxford, North Thames, North West, South and West, South Thames, Trent, and West Midlands regions; and by the Norwegian Research Council and Ministry of Health and Social Affairs in Norway. The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Office Home and Health Department. The views expressed are those of the authors and not necessarily the funding bodies.

Conflict of interest: None.

    References

  1. Eddy DM. Clinical policies and the quality of clinical practice. N Engl J Med 1982; 307: 343-347[Medline].
  2. Grimshaw JM, Freemantle N, Langhorne P, Song F. Complexity and systematic reviews: report to the US Congress Office of Technology Assessment. Washington, DC: Office of Technology Assessment , 1995.
  3. Mulrow CD. Rationale for systematic reviews. BMJ 1994; 309: 597-599[Free Full Text].
  4. Bero L, Grilli R, Grimshaw JM, Harvey E, Oxman AD, eds. Effective professional practice and organisation of care module, Cochrane Database of Systematic Reviews. The Cochrane Library. The Cochrane Collaboration; Issue 4. Oxford: Update Software; 1997. 
  5. Implementing clinical guidelines: can guidelines be used to improve clinical practice? Effective Health Care 1994; No 8. 
  6. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993; 342: 1317-1322[Medline].
  7. Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci 1993; 703: 123-131.
  8. Oxman AD. Checklists for review articles. BMJ 1994; 309: 648-651[Free Full Text].
  9. Lomas J. Words without action? The production, dissemination, and impact of consensus recommendations. Annu Rev Public Health 1991; 12: 41-65[Medline].
  10. Grilli R, Lomas J. Evaluating the message: the relationship between compliance rate and the subject of a practice guideline. Med Care 1994; 32: 202-213[Medline].
  11. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to help health care professionals deliver services more effectively or efficiently. Can Med Assoc J 1995; 153: 1423-1431[Abstract].
  12. Beaudry JS. The effectiveness of continuing medical education: a quantitative synthesis. J Continuing Education Health Professions 1989; 9: 285-307.
  13. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995; 274: 700-705[Abstract].
  14. Mugford M, Banfield P, O'Hanlon M. Effects of feedback of information on clinical practice: a review. BMJ 1991; 303: 398-402.
  15. Buntinx F, Winkens R, Grol R, Knottnerus JA. Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders: a review. Fam Pract 1993; 10: 219-228[Abstract/Free Full Text].
  16. Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med 1994; 120: 135-142[Abstract/Free Full Text].
  17. Austin SM, Balas EA, Mitchell JA, Ewigman BG. Effect of physician reminders on preventive care: meta-analysis of randomized clinical trials. Proceedings---the Annual Symposium on Computer Applications in Medical Care 1994;121-4.
  18. Wensing M, Grol R. Single and combined strategies for implementing changes in primary care: a literature review. Int J Quality Health Care 1994; 6: 115-132.[Abstract/Free Full Text]
  19. Waddell DL. The effects of continuing education on nursing practice: a meta-analysis. J Continuing Education Nurs 1991; 22: 113-118.
  20. Yano EM, Fink A, Hirsch SH, Robbins AS, Rubenstein LV. Helping practices reach primary care goals: lessons from the literature. Arch Intern Med 1995; 155: 1146-1156[Abstract].
  21. Soumerai SB, McLaughlin TJ, Avorn J. Improving drug prescribing in primary care: a critical analysis of the experimental literature. Milbank Q 1989; 67: 268-317[Medline].
  22. Lomas J, Haynes RB. A taxonomy and critical review of tested strategies for the application of clinical practice recommendations: from "official" to "individual" clinical policy. Am J Prev Med 1987; 4: 77-94.
  23. Gyorkos TW, Tannenbaum TN, Abrahamowicz M, Bédard L, Carsley J, Franco ED, et al. Evaluation of the effectiveness of immunization delivery methods. Can J Public Health 1994; 85(suppl 1): 14-30S.
  24. Mandleblatt J, Kanetsky PA. Effectiveness of interventions to enhance physician screening for breast cancer. J Fam Pract 1995; 40: 162-171[Medline].
  25. Silagy C, Lancaster T, Gray S, Fowler G. The effectiveness of training health professionals to provide smoking cessation interventions: systematic review of randomised controlled trials. Qual Health Care 1995; 3: 193-198[Abstract].
  26. Lomas J, Anderson GM, Domnick-Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med 1989; 321: 1306-1311[Abstract].
  27. Whiting-O'Keefe QE, Henke C, Simborg DW. Choosing the correct unit of analysis in medical care experiments. Med Care 1984; 22: 1101-1114[Medline].
  28. Grol R. Implementing guidelines in general practice care. Qual Health Care 1992; 1: 184-191[Medline].
  29. Donner A, Birkett N, Buck C. Randomisation by cluster: sample size requirements and analysis. Am J Epidemiol 1981; 114: 906-914[Abstract/Free Full Text].
  30. NHS Research and Development Programme. Methods to promote the implementation of research findings in the NHS: priorities for evaluation. Leeds: Department of Health , 1995.
  31. Freemantle N, Grilli R, Grimshaw JM, Oxman A. Implementing the findings of medical research: the Cochrane Collaboration on effective professional practice. Qual Health Care 1995; 4: 45-47[Medline].

© BMJ 1998

Related Articles

Computerised evidence based guidelines in primary care: Authors' reply
Martin Eccles, Nikki Rousseau, Elaine McColl, John Newton, and Jeremy Grimshaw
BMJ 2003 326: 1088. [Extract] [Full Text]

Between the idea and the reality falls the shadow
BMJ 1998 317: 0. [Full Text]

Between the idea and the reality falls the shadow
BMJ 1998 317: 0. [Full Text]

This article has been cited by other articles:

  • Gossec, L, Dougados, M, Phillips, C, Hammoudeh, M, de Vlam, K, Pavelka, K, Pham, T, Braun, J, Sieper, J, Olivieri, I, van der Heijde, D, Collantes, E, Stone, M, Kvien, T K, on behalf of ASAS (ASsessment in AS international, (2008). Dissemination and evaluation of the ASAS/EULAR recommendations for the management of ankylosing spondylitis: results of a study among 1507 rheumatologists. Ann Rheum Dis 67: 782-788 [Abstract] [Full text]  
  • Althabe, F., Buekens, P., Bergel, E., Belizan, J. M., Campbell, M. K., Moss, N., Hartwell, T., Wright, L. L., the Guidelines Trial Group, (2008). A Behavioral Intervention to Improve Obstetrical Care. NEJM 358: 1929-1940 [Abstract] [Full text]  
  • Thompson, D. S, Moore, K. N, Estabrooks, C. A (2008). Increasing research use in nursing: implications for clinical educators and managers. Evid. Based Nurs. 11: 35-39 [Full text]  
  • Agley, J., Gassman, R. (2008). A State-Based Model of Prevention: Indiana's Example. Health Promot Pract 9: 199-204 [Abstract]  
  • Kramer, T. L., Miller, T. L., Phillips, S. D., Robbins, J. M. (2008). Quality of Mental Health Care for Depressed Adolescents. American Journal of Medical Quality 23: 96-104 [Abstract]  
  • Sullivan, S., Thompson, A. (2008). Bridging the gap between the university and the NHS: health services research in psychiatry. Psychiatr. Bull. 32: 146-149 [Full text]  
  • Parra, E., Ramos, R., Betriu, A., Paniagua, J., Belart, M., Martinez, T. (2008). Effect of a quality improvement strategy on several haemodialysis outcomes. Nephrol Dial Transplant 0: gfn116v1-gfn116 [Abstract] [Full text]  
  • Slora, E. J., Steffes, J. M., Harris, D., Clegg, H. W., Norton, D., Darden, P. M., Sullivan, S. A., Wasserman, R. C. (2008). Improving Pediatric Practice Immunization Rates Through Distance-Based Quality Improvement: A Feasibility Trial from PROS. CLIN PEDIATR 47: 25-36 [Abstract]  
  • Boluyt, N., Tjosvold, L., Lefebvre, C., Klassen, T. P., Offringa, M. (2008). Usefulness of Systematic Review Search Strategies in Finding Child Health Systematic Reviews in MEDLINE. Arch Pediatr Adolesc Med 162: 111-116 [Abstract] [Full text]  
  • Jack, L. Jr (2007). Exploring Healthy Coping Behaviors in Diabetes Self-management: A Commentary. The Diabetes Educator 33: 1104-1106 [Full text]  
  • Brownson, R. C., Ballew, P., Brown, K. L., Elliott, M. B., Haire-Joshu, D., Heath, G. W., Kreuter, M. W. (2007). The Effect of Disseminating Evidence-Based Interventions That Promote Physical Activity to Health Departments. Am. J. Public Health 97: 1900-1907 [Abstract] [Full text]  
  • Mourad, S.M., Hermens, R.P.M.G., Nelen, W.L.D.M., Braat, D.D.M., Grol, R.P.T.M., Kremer, J.A.M. (2007). Guideline-based development of quality indicators for subfertility care. Hum Reprod 22: 2665-2672 [Abstract] [Full text]  
  • Matthys, J., De Meyere, M., van Driel, M. L., De Sutter, A. (2007). Differences Among International Pharyngitis Guidelines: Not Just Academic. Ann Fam Med 5: 436-443 [Abstract] [Full text]  
  • Tu, S. W., Campbell, J. R., Glasgow, J., Nyman, M. A., McClure, R., McClay, J., Parker, C., Hrabak, K. M., Berg, D., Weida, T., Mansfield, J. G., Musen, M. A., Abarbanel, R. M. (2007). The SAGE Guideline Model: Achievements and Overview. J. Am. Med. Inform. Assoc. 14: 589-598 [Abstract] [Full text]  
  • Zolotor, A. J., Randolph, G. D., Johnson, J. K., Wegner, S., Edwards, L., Powell, C., Esporas, M. H. (2007). Effectiveness of a Practice-Based, Multimodal Quality Improvement Intervention for Gastroenteritis Within a Medicaid Managed Care Network. Pediatrics 120: e644-e650 [Abstract] [Full text]  
  • Rhodes, E. T., Ebbeling, C. B., Meyers, A. F., Bayerl, C. T., Ooi, W. L., Bettencourt, M. F., Ludwig, D. S. (2007). Pediatric Obesity Management: Variation by Specialty and Awareness of Guidelines. CLIN PEDIATR 46: 491-504 [Abstract]  
  • Kontos, P. C., Naglie, G. (2007). Expressions of Personhood in Alzheimer's Disease: An Evaluation of Research-Based Theatre as a Pedagogical Tool. Qual Health Res 17: 799-811 [Abstract]  
  • Hasselaar, J. G. J., Reuzel, R. P. B., Verhagen, S. C. A. H. H. V. M., de Graeff, A., Vissers, K. C. P., Crul, B. J. P. (2007). Improving Prescription in Palliative Sedation: Compliance With Dutch Guidelines. Arch Intern Med 167: 1166-1171 [Abstract] [Full text]  
  • Cleophas, T. J., Zwinderman, A. H. (2007). Meta-Analysis. Circulation 115: 2870-2875 [Full text]  
  • Nelen, W.L.D.M., Hermens, R.P.M.G., Mourad, S.M., Haagen, E.C., Grol, R.P.T.M., Kremer, J.A.M. (2007). Monitoring reproductive health in Europe: what are the best indicators of reproductive health? A need for evidence-based quality indicators of reproductive health care. Hum Reprod 22: 916-918 [Abstract] [Full text]  
  • Aittasalo, M., Miilunpalo, S., Stahl, T., Kukkonen-Harjula, K. (2007). From innovation to practice: initiation, implementation and evaluation of a physician-based physical activity promotion programme in Finland. HEALTH PROMOT INT 22: 19-27 [Abstract] [Full text]  
  • Hamilton, W., Russell, D., Stabb, C., Seamark, D., Campion-Smith, C., Britten, N. (2007). The effect of patient self-completion agenda forms on prescribing and adherence in general practice: a randomized controlled trial. Fam Pract 24: 77-83 [Abstract] [Full text]  
  • Bain, K. T. (2007). Barriers and Strategies to Influencing Physician Behavior. American Journal of Medical Quality 22: 5-7  
  • Astin, J. A., Soeken, K., Sierpina, V. S., Clarridge, B. R. (2006). Barriers to the Integration of Psychosocial Factors in Medicine: Results of a National Survey of Physicians. J Am Board Fam Med 19: 557-565 [Abstract] [Full text]  
  • Payne, N. R., LaCorte, M., Sun, S., Karna, P., Lewis-Hunstiger, M., Goldsmith, J. P., on behalf of the Breathsavers Group, (2006). Evaluation and Development of Potentially Better Practices to Reduce Bronchopulmonary Dysplasia in Very Low Birth Weight Infants. Pediatrics 118: S65-S72 [Abstract] [Full text]  
  • Alvarez, M. P., Agra, Y. (2006). Systematic review of educational interventions in palliative care for primary care physicians.. Palliat Med 20: 673-683 [Abstract]  
  • Asadoorian, J., Locker, D. (2006). The impact of quality assurance programming: a comparison of two canadian dental hygienist programs.. J Dent Educ 70: 965-971 [Abstract] [Full text]  
  • Ginexi, E. M., Hilton, T. F. (2006). What's Next for Translation Research?. Eval Health Prof 29: 334-347 [Abstract]  
  • Hroscikoski, M. C., Solberg, L. I., Sperl-Hillen, J. M., Harper, P. G., McGrail, M. P., Crabtree, B. F. (2006). Challenges of Change: A Qualitative Study of Chronic Care Model Implementation. Ann Fam Med 4: 317-326 [Abstract] [Full text]  
  • Thamlikitkul, V., Indranoi, A. (2006). Switching from heparinized saline flush to normal saline flush for maintaining peripheral venous catheter patency.. Int J Qual Health Care 18: 183-185 [Abstract] [Full text]  
  • Kattan, M., Crain, E. F., Steinbach, S., Visness, C. M., Walter, M., Stout, J. W., Evans, R. III, Smartt, E., Gruchalla, R. S., Morgan, W. J., O'Connor, G. T., Mitchell, H. E. (2006). A randomized clinical trial of clinician feedback to improve quality of care for inner-city children with asthma.. Pediatrics 117: e1095-e1103 [Abstract] [Full text]  
  • Zielinski, J., Bednarek, M., Gorecka, D., Viegi, G., Hurd, S. S., Fukuchi, Y., Lai, C. K. W., Ran, P. X., Ko, F. W. S., Liu, S. M., Zheng, J. P., Zhong, N. S., Ip, M. S. M., Vermeire, P. A. (2006). Increasing COPD awareness.. Eur Respir J 27: 833-852 [Full text]  
  • Sondergaard, J., Hansen, D. G., Aarslev, P., Munck, A. P (2006). A multifaceted intervention according to the Audit Project Odense method improved secondary prevention of ischemic heart disease: a randomised controlled trial. Fam Pract 23: 198-202 [Abstract] [Full text]  
  • Feifer, C., Ornstein, S. M., Jenkins, R. G., Wessell, A., Corley, S. T., Nemeth, L. S., Roylance, L., Nietert, P. J., Liszka, H. (2006). The logic behind a multimethod intervention to improve adherence to clinical practice guidelines in a nationwide network of primary care practices.. Eval Health Prof 29: 65-88 [Abstract]  
  • Maier, R. V. (2006). What the surgeon of tomorrow needs to know about evidence-based surgery.. Arch Surg 141: 317-323 [Full text]  
  • Yealy, D. M., Auble, T. E., Stone, R. A., Lave, J. R., Meehan, T. P., Graff, L. G., Fine, J. M., Obrosky, D. S., Mor, M. K., Whittle, J., Fine, M. J. (2005). Effect of Increasing the Intensity of Implementing Pneumonia Guidelines: A Randomized, Controlled Trial. ANN INTERN MED 143: 881-894 [Abstract] [Full text]  
  • Lowcay, B., McIntyre, E. (2005). Research posters--the way to display. BMJ CAREER FOCUS 331: 251-252 [Full text]  
  • Haagen, E.C., Nelen, W.L.D.M., Hermens, R.P.M.G., Braat, D.D.M., Grol, R.P.T.M., Kremer, J.A.M. (2005). Barriers to physician adherence to a subfertility guideline. Hum Reprod 20: 3301-3306 [Abstract] [Full text]  
  • Muijrers, P. E., Grol, R. P., Sijbrandij, J., Janknegt, R., Knottnerus, J A. (2005). Differences in prescribing between GPs. Impact of the cooperation with pharmacists and impact of visits from pharmaceutical industry representatives. Fam Pract 22: 624-630 [Abstract] [Full text]  
  • Salaun, P.-Y., Couturaud, F., Lacut, K., Nonent, M., Gut-Gobert, C., Le Gal, G., Guillo, P., Jouquan, J., Mottier, D., Leroyer, C. (2005). Management of suspected venous thromboembolism: the impact of a multifaceted intervention. Int J Qual Health Care 17: 433-438 [Abstract] [Full text]  
  • Willigendael, E. M, Teijink, J. A., Bartelink, M.-L., Langenberg, M., Welten, R. J., Buller, H. R, Prins, M. H (2005). Combined training of GPs and practice-assistants on peripheral arterial disease: positive effects after six months. Fam Pract 22: 520-522 [Abstract] [Full text]  
  • Sierra, R., Benitez, E., Leon, C., Rello, J. (2005). Prevention and Diagnosis of Ventilator-Associated Pneumonia: A Survey on Current Practices in Southern Spanish ICUs. Chest 128: 1667-1673 [Abstract] [Full text]  
  • Wall, R J, Ely, E W, Elasy, T A, Dittus, R S, Foss, J, Wilkerson, K S, Speroff, T (2005). Using real time process measurements to reduce catheter related bloodstream infections in the intensive care unit. Qual Saf Health Care 14: 295-302 [Abstract] [Full text]  
  • Lehoux, P., Denis, J.-L., Tailliez, S., Hivon, M. (2005). Dissemination of Health Technology Assessments: Identifying the Visions Guiding an Evolving Policy Innovation in Canada. Journal of Health Politics, Policy and Law 30: 603-642 [Abstract]  
  • Mazieres, B, Scmidely, N, Hauselmann, H J, Martin-Mola, E, Serni, U, Verbruggen, A A, Le Bars, M (2005). Level of acceptability of EULAR recommendations for the management of knee osteoarthritis by practitioners in different European countries. Ann Rheum Dis 64: 1158-1164 [Abstract] [Full text]  
  • Smith, B J, Dalziel, K, McElroy, H J, Ruffin, R E, Frith, P A, McCaul, K A, Cheok, F (2005). Barriers to success for an evidence-based guideline for chronic obstructive pulmonary disease. Chronic Respiratory Disease 2: 121-131 [Abstract]  
  • Krumholz, H. M., Peterson, E. D., Ayanian, J. Z., Chin, M. H., DeBusk, R. F., Goldman, L., Kiefe, C. I., Powe, N. R., Rumsfeld, J. S., Spertus, J. A., Weintraub, W. S. (2005). Report of the National Heart, Lung, and Blood Institute Working Group on Outcomes Research in Cardiovascular Disease. Circulation 111: 3158-3166 [Abstract] [Full text]  
  • Osser, D. N., Patterson, R. D., Levitt, J. J. (2005). Guidelines, Algorithms, and Evidence-Based Psychopharmacology Training for Psychiatric Residents. Acad. Psychiatry 29: 180-186 [Abstract] [Full text]  
  • Bekkering, G. E, van Tulder, M. W, Hendriks, E. J., Koopmanschap, M. A, Knol, D. L, Bouter, L. M, Oostendorp, R. A. (2005). Implementation of Clinical Guidelines on Physical Therapy for Patients With Low Back Pain: Randomized Trial Comparing Patient Outcomes After a Standard and Active Implementation Strategy. ptjournal 85: 544-555 [Abstract] [Full text]  
  • Gattellari, M., Donnelly, N., Taylor, N., Meerkin, M., Hirst, G., Ward, J. E (2005). Does 'peer coaching' increase GP capacity to promote informed decision making about PSA screening? A cluster randomised trial. Fam Pract 22: 253-265 [Abstract] [Full text]  
  • Tinmouth, A., MacDougall, L., Fergusson, D., Amin, M., Graham, I. D., Hebert, P. C., Wilson, K. (2005). Reducing the Amount of Blood Transfused: A Systematic Review of Behavioral Interventions to Change Physicians' Transfusion Practices. Arch Intern Med 165: 845-852 [Abstract] [Full text]  
  • Kawamoto, K., Houlihan, C. A, Balas, E A., Lobach, D. F (2005). Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 330: 765- [Abstract] [Full text]  
  • Bekkering, G E, Hendriks, H J M, van Tulder, M W, Knol, D L, Hoeijenbos, M, Oostendorp, R A B, Bouter, L M (2005). Effect on the process of care of an active strategy to implement clinical guidelines on physiotherapy for low back pain: a cluster randomised controlled trial. Qual Saf Health Care 14: 107-112 [Abstract] [Full text]  
  • Mol, P. G. M., Wieringa, J. E., NannanPanday, P. V., Gans, R. O. B., Degener, J. E., Laseur, M., Haaijer-Ruskamp, F. M. (2005). Improving compliance with hospital antibiotic guidelines: a time-series intervention analysis. J Antimicrob Chemother 55: 550-557 [Abstract] [Full text]  
  • Ozer, E. M., Adams, S. H., Lustig, J. L., Gee, S., Garber, A. K., Gardner, L. R., Rehbein, M., Addison, L., Irwin, C. E. Jr (2005). Increasing the Screening and Counseling of Adolescents for Risky Health Behaviors: A Primary Care Intervention. Pediatrics 115: 960-968 [Abstract] [Full text]  
  • Wei, E. K., Ryan, C. T., Dietrich, A. J., Colditz, G. A. (2005). Improving Colorectal Cancer Screening by Targeting Office Systems in Primary Care Practices: Disseminating Research Results Into Clinical Practice. Arch Intern Med 165: 661-666 [Abstract] [Full text]  
  • Cheng, D. C. H., Martin, J. E. (2005). Raising the Bar: A Primer On Evidence-Based Decision-Making. SEMIN CARDIOTHORAC VASC ANESTH 9: 1-4  
  • von Gunten, V., Troillet, N., Beney, J., Boubaker, K., Luthi, J.-C., Taffe, P., Reymond, J.-P. (2005). Impact of an interdisciplinary strategy on antibiotic use: a prospective controlled study in three hospitals. J Antimicrob Chemother 55: 362-366 [Abstract] [Full text]  
  • Ringel, M. D., Mazzaferri, E. L. (2005). Subclinical Thyroid Dysfunction--Can There Be a Consensus about the Consensus?. J. Clin. Endocrinol. Metab. 90: 588-590 [Full text]  
  • Gardner, M. J., Brophy, R. H., Demetrakopoulos, D., Koob, J., Hong, R., Rana, A., Lin, J. T., Lane, J. M. (2005). Interventions to Improve Osteoporosis Treatment Following Hip Fracture. A Prospective, Randomized Trial. JBJS 87: 3-7 [Abstract] [Full text]  
  • Shojania, K. G., Grimshaw, J. M. (2005). Evidence-Based Quality Improvement: The State Of The Science. Health Aff (Millwood) 24: 138-150 [Abstract] [Full text]  
  • Reidenberg, M. M, Walley, T. (2004). The pros and cons of essential medicines for rich countries. BMJ 329: 1172-1172 [Full text]  
  • BootsMiller, B. J., Yankey, J. W., Flach, S. D., Ward, M. M., Vaughn, T. E., Welke, K. F., Doebbeling, B. N. (2004). Classifying the Effectiveness of Veterans Affairs Guideline Implementation Approaches. American Journal of Medical Quality 19: 248-254 [Abstract]  
  • Rowlands, P. (2004). The NICE schizophrenia guidelines: the challenge of implementation. Adv. Psychiatr. Treat. 10: 403-412 [Abstract] [Full text]  
  • Verstappen, W. H. J. M., van Merode, F., Grimshaw, J., Dubois, W. I., Grol, R. P. T. M., van Der Weijden, T. (2004). Comparing cost effects of two quality strategies to improve test ordering in primary care: a randomized trial. Int J Qual Health Care 16: 391-398 [Abstract] [Full text]  
  • Akbari, A., Swedko, P. J., Clark, H. D., Hogg, W., Lemelin, J., Magner, P., Moore, L., Ooi, D. (2004). Detection of Chronic Kidney Disease With Laboratory Reporting of Estimated Glomerular Filtration Rate and an Educational Program. Arch Intern Med 164: 1788-1792 [Abstract] [Full text]  
  • Majumdar, S. R., Rowe, B. H., Folk, D., Johnson, J. A., Holroyd, B. H., Morrish, D. W., Maksymowych, W. P., Steiner, I. P., Harley, C. H., Wirzba, B. J., Hanley, D. A., Blitz, S., Russell, A. S. (2004). A Controlled Trial To Increase Detection and Treatment of Osteoporosis in Older Patients with a Wrist Fracture. ANN INTERN MED 141: 366-373 [Abstract] [Full text]  
  • Goldstein, M. K., Coleman, R. W., Tu, S. W., Shankar, R. D., O'Connor, M. J., Musen, M. A., Martins, S. B., Lavori, P. W., Shlipak, M. G., Oddone, E., Advani, A. A., Gholami, P., Hoffman, B. B. (2004). Translating Research into Practice: Organizational Issues in Implementing Automated Decision Support for Hypertension in Three Medical Centers. J. Am. Med. Inform. Assoc. 11: 368-376 [Abstract] [Full text]  
  • Simpson, L. (2004). Lost In Translation? Reflections On The Role Of Research In Improving Health Care For Children. Health Aff (Millwood) 23: 125-130 [Abstract] [Full text]  
  • Gaspoz, J.-M. (2004). Quality of in-hospital care in acute coronary syndromes: it is time to close the gap. Int J Qual Health Care 16: 273-274 [Full text]  
  • Scott, I. A., Denaro, C. P., Hickey, A. C., Bennett, C., Mudge, A. M., Sanders, D. C., Thiele, J., Flores, J. L. (2004). Optimising care of acute coronary syndromes in three Australian hospitals. Int J Qual Health Care 16: 275-284 [Abstract] [Full text]  
  • Roumie, C. L., Grogan, E. L., Falbe, W., Awad, J., Speroff, T., Dittus, R. S., Elasy, T. A. (2004). A Three-Part Intervention To Change the Use of Hormone Replacement Therapy in Response to New Evidence. ANN INTERN MED 141: 118-125 [Abstract] [Full text]  
  • Vogtlander, N. P. J., van Kasteren, M. E. E., Natsch, S., Kullberg, B.-J., Hekster, Y. A., van der Meer, J. W. M. (2004). Improving the Process of Antibiotic Therapy in Daily Practice: Interventions to Optimize Timing, Dosage Adjustment to Renal Function, and Switch Therapy. Arch Intern Med 164: 1206-1212 [Abstract] [Full text]  
  • Buekens, P., Keusch, G., Belizan, J., Bhutta, Z. A. (2004). Evidence-Based Global Health. JAMA 291: 2639-2641 [Full text]  
  • Bours, G. J. J. W., Halfens, R. J. G., Candel, M. J. J. M., Grol, R. T. P. M., Abu-Saad, H. H. (2004). A pressure ulcer audit and feedback project across multi-hospital settings in the Netherlands. Int J Qual Health Care 16: 211-218 [Abstract] [Full text]  
  • Russell, J., Greenhalgh, T., Boynton, P., Rigby, M. (2004). Soft networks for bridging the gap between research and practice: illuminative evaluation of CHAIN. BMJ 328: 1174- [Abstract] [Full text]  
  • Meehan, T. P., Tate, J. P., Holmboe, E. S., Teeple, E. A., Elwell, A., Meehan, R. R., Petrillo, M. K., Huot, S. J. (2004). A Collaborative Initiative to Improve the Care of Elderly Medicare Patients With Hypertension. American Journal of Medical Quality 19: 103-111 [Abstract]  
  • De Maeseneer, J. M., De Sutter, A. (2004). Why Research in Family Medicine? A Superfluous Question. Ann Fam Med 2: S17-S22 [Abstract] [Full text]  
  • Lam, C. L. K. (2004). The 21st Century: The Age of Family Medicine Research?. Ann Fam Med 2: S50-S54 [Abstract] [Full text]  
  • Qvist, P., Rasmussen, L., Bonnevie, B., Gjorup, T. (2004). Repeated measurements of generic indicators: a Danish national program to benchmark and improve quality of care. Int J Qual Health Care 16: 141-148 [Abstract] [Full text]  
  • Butzlaff, M, Vollmar, H., Floer, B, Koneczny, N, Isfort, J, Lange, S (2004). Learning with computerized guidelines in general practice?: A randomized controlled trial. Fam Pract 21: 183-188 [Abstract] [Full text]  
  • Martin, C. M., Doig, G. S., Heyland, D. K., Morrison, T., Sibbald, W. J. (2004). Multicentre, cluster-randomized clinical trial of algorithms for critical-care enteral and parenteral therapy (ACCEPT). CMAJ 170: 197-204 [Abstract] [Full text]  
  • Smith, B J, Cheok, F, Heard, A R, Esterman, A J, Southcott, A M, Antic, R, Frith, P A, Hender, K, Ruffin, R E (2004). Impact on readmission rates and mortality of a chronic obstructive pulmonary disease inpatient management guideline. Chronic Respiratory Disease 1: 17-28 [Abstract]  
  • Beaulieu, M.-D., Brophy, J., Jacques, A., Blais, R., Battista, R., Lebeau, R. (2004). Drug treatment of stable angina pectoris and mass dissemination of therapeutic guidelines: a randomized controlled trial. QJM 97: 21-31 [Abstract] [Full text]  
  • Boyle, R., Solberg, L. I. (2004). Is Making Smoking Status a Vital Sign Sufficient to Increase Cessation Support Actions in Clinical Practice?. Ann Fam Med 2: 22-25 [Abstract] [Full text]  
  • Kiessling, A. (2004). Participatory learning: a Swedish perspective. Heart 90: 113-116 [Full text]  
  • Vale, M. J., Jelinek, M. V., Best, J. D., Dart, A. M., Grigg, L. E., Hare, D. L., Ho, B. P., Newman, R. W., McNeil, J. J. (2003). Coaching patients On Achieving Cardiovascular Health (COACH): A Multicenter Randomized Trial in Patients With Coronary Heart Disease. Arch Intern Med 163: 2775-2783 [Abstract] [Full text]  
  • Feldstein, A. C., Nichols, G. A., Elmer, P. J., Smith, D. H., Aickin, M., Herson, M. (2003). Older Women with Fractures: Patients Falling Through the Cracks of Guideline-Recommended Osteoporosis Screening and Treatment. JBJS 85: 2294-2302 [Abstract] [Full text]  
  • Scheel, I B, Hagen, K B, Oxman, A D (2003). The unbearable lightness of healthcare policy making: a description of a process aimed at giving it some weight. J. Epidemiol. Community Health 57: 483-487 [Abstract] [Full text]  
  • Dopson, S., Locock, L., Gabbay, J., Ferlie, E., Fitzgerald, L. (2003). Evidence-Based Medicine and the Implementation Gap. Health (London) 7: 311-330 [Abstract]  
  • Young, J. M., Hollands, M. J., Ward, J., Holman, C. D'A. J. (2003). Role for Opinion Leaders in Promoting Evidence-Based Surgery. Arch Surg 138: 785-791 [Abstract] [Full text]  
  • Steinberg, E. P. (2003). Improving the Quality of Care -- Can We Practice What We Preach?. NEJM 348: 2681-2683 [Full text]  
  • Stryer, D., Clancy, C. (2003). Boosting performance measure for measure. BMJ 326: 1278-1279 [Full text]  
  • Frijling, B D, Lobo, C M, Hulscher, M E J L, Akkermans, R P, van Drenth, B B, Prins, A, van der Wouden, J C, Grol, R P T M (2003). Intensive support to improve clinical decision making in cardiovascular care: a randomised controlled trial in general practice. Qual Saf Health Care 12: 181-187 [Abstract] [Full text]  
  • Foy, R, Warner, P (2003). About time: diagnostic guidelines that help clinicians. Qual Saf Health Care 12: 205-209 [Abstract] [Full text]  
  • van Bokhoven, M A, Kok, G, van der Weijden, T (2003). Designing a quality improvement intervention: a systematic approach. Qual Saf Health Care 12: 215-220 [Abstract] [Full text]  
Tools
Services
Online poll
Find out more

Rapid responses for this article

There are no rapid responses for this article.

Student BMJ

Risk of surgery for inflammatory bowel disease: record linkage studies

What can you learn from this BMJ paper? Read Leanne Tite's Paper+

www.student.bmj.com

Listen to the latest BMJ Interview