Education and debate

Finding information on clinical effectiveness

Julie Glanville, information service manager, ^a Margaret Haines, principal adviser, ^b Ione Auston, librarian. ^c

^a NHS Centre for Reviews and Dissemination, University of York, York Y01 5DD, ^b Library and Information Commission, London W1V 4BH, ^c National Information Center on Health Services Research and Health Care Technology, National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, USA

Correspondence to: Ms Glanville jmg1{at}york.ac.uk

Series editors: Andrew Haines and Anna Donald

There is increasing pressure on healthcare professionals to ensure that their practice is based on evidence from good quality research, such as randomised controlled trials or, preferably, systematic reviews of randomised controlled trials and trials of other study designs. This pressure comes from various sources. The evidence based healthcare movement encourages a questioning and reflective approach to clinical practice and emphasises the importance of lifelong learning. Thus, good access to research based evidence is necessary. Many governments are encouraging the development of evidence based medicine because its advantages are understood, especially in terms of improved efficiency in the delivery of health care through the identification of effective treatments. ^{1 2} There are also indications that legal decisions may take account of whether research evidence and clinical guidelines were adhered to. ^{3 4} Better informed consumers may provide another incentive for clinicians to be more aware of research findings. Clinicians will need to be able to access information on clinical effectiveness in order to improve the quality of care and to stay well informed on developments in specialist areas. We examine the resources that are already available to clinicians, strategies for finding and filtering information, and ways of improving dissemination.

	Evidence based information already available

In the 1990s evidence from research has become more easily available. In part this has been due to the development of programmes for assessing health technology and to the growth in systematic reviews. Systematic reviews evaluate primary evidence and the effectiveness of particular interventions. They necessarily take time to complete but a useful compilation of reviews is available in The Cochrane Library and there are also reports from technology assessment agencies such as the Agency for Health Care Policy and Research in the United States or in England the Department of Health's health technology assessment programme. The publications and databases in the box present evidence on effectiveness, often in a summarised form suitable for the busy clinician or policymaker. However, important problems remain, such as how to increase awareness of what information is available and how to provide clinicians with information when they need it.

Collections of systematic reviews and critical appraisals of primary research are valuable sources of evaluated research. The proliferation of these collections is creating its own information explosion; this is a serious problem that needs to be addressed. Because there is no single comprehensive index to all the material described in the box several searches through both paper journals and electronic services may be required to locate relevant information. It may also be necessary to obtain copies of the original publication. These are disincentives to searching for and obtaining research evidence. Information technology may eventually provide a more streamlined way of dealing with this explosion of information, perhaps in the form of world wide web interfaces that provide links to a range of evidence based information services that filter publications for quality, or by providing access to the full text of publications. Biomednet is one model of this type of service. It offers a range of full text resources with free Medline access, discussion facilities, and virtual meeting rooms. Biomednet is beginning to highlight important papers that have been cited and evaluated by expert reviewers as a means of filtering papers for quality.⁵

The resources in the box provide information that has been evaluated and filteredthat is, they highlight the best quality studies from the mass of available literature. However, research based answers to many questions of effectiveness are not yet available in such time saving, value added forms. Clinicians may still need to search indexes and abstracts of published literature. For several years it has been possible for clinicians to search Medline using software such as Grateful Med, and its world wide web interface, internet Grateful Med. This has provided access to a large body of peer reviewed studies that are mostly unsynthesised and unevaluated. There are tools to help searchers identify the types of studies that are more likely to provide high quality information on clinical effectiveness, such as systematic reviews or randomised controlled trials. ^{6 7} Once the original papers have been retrieved there are checklists that, together with training in critical appraisal skills, can be used to assess the rigour and validity of such studies.^8-10

Although Medline is a rich resource, access is increasingly required to a wider range of material than it presently indexes. The US National Library of Medicine and the American Hospital Association have recently launched the HealthSTAR database which seeks to provide expanded access to both non-clinical information (on topics such as healthcare administration, economics, and planning) and non-journal information (such as reports, meeting abstracts, and chapters from books) that is not available in journals.¹¹ The National Library of Medicine has recently announced that access to Medline and HealthSTAR through internet Grateful Med and access to Medline through the PubMed interface will be free.¹¹ Other databases that cover specific clinical areas, specific types of publications, and non-English language material should also be used. Tools such as search strategies and single interfaces, like PubMed, are required to enhance access to a range of such databases.

	Strategies for finding and filtering information

Training and practice are required to search information services and navigate the internet effectively, but other options are available which may help clinicians cope with the challenges of finding information. Locating, appraising, and exploiting resources, both print and electronic, has typically been the role of the librarian or information professional. Increasingly, clinicians are finding that librarians can not only help them locate information in answer to a particular question but also can help to keep their knowledge up to date by presenting selections of important new evidence in the form of paper or electronic bulletins.

The value of library and information support has been demonstrated on both sides of the Atlantic. Trained librarians are often more effective than physicians in filtering papers for quality.¹² Some American studies have shown that library support not only contributes to lower patient care costs by decreasing the number of admissions, length of stay, and number of procedures but also contributes to a higher quality of care in terms of patient advice, improved decision making, and savings in time. ^{13 14} A similar study in the United Kingdom found that library services had a positive impact on the continuing education of hospital doctors.¹⁵

Not all clinicians have the time to visit libraries, and new models have emerged for delivering library support directly to hospital wards and departments. ^{16 17} In the United States, the National Network of Libraries of Medicine provides outreach services to general practitioners (and, more recently, to public health professionals); in the United Kingdom the BMA library offers an electronic outreach service to members. ^{18 19} Also in the United Kingdom, the Oxford PRISE (primary care sharing the evidence) project is developing a model whereby general practitioners' computers are linked to a central computer that provides access to a range of databases; in this model the general practitioners can also request librarians to follow up particular questions in more detail.²⁰ Librarians are increasingly asked to provide training in information skills as part of courses in evidence based medicine offered to NHS staff.

The development of primary care based services presents a challenge to librarians; they must become better trained to deal with a wider range of inquiries, to evaluate and synthesise evidence, and to present selected information through innovative delivery systems. Clearly initiatives such as the Oxford Health Libraries' training programme, known as the "librarian of the 21st century," is a model for other library networks.²¹ Similar initiatives under development in the United States include the National Information Center on Health Services Research and Health Care Technology, web based training materials that are not copyrighted and can be modified to suit the user,²² and training programmes for librarians sponsored by the National Library of Medicine in subjects such as medical informatics.

	Improving dissemination

For information to be accessible it must be packaged and published in formats that promote easy identification and encourage use. Evidence based information is becoming easier to find: structured abstracts in articles in journals make it easier to identify the methodology of a study and its potential reliability. Innovations, such as the BMJ's key messages boxes, make it easier to identify the important points of research. Journal editors have an important role in encouraging authors to provide informative abstracts and in ensuring that researchers' conclusions are supported by their paper's results. However, the benefits of clearer labelling may be undermined if current buzz words, such as "effective" and "evidence based," are adopted and used incorrectly or inaccurately so that previously useful labels become meaningless.

Organisations that produce recommendations on policy and clinical guidelines are finding it necessary to make their guidelines more explicitly evidence based, both by using research evidence to develop their guidelines and in stating the level of evidence on which the guidance is based. ^{23 24} It would be easier and quicker to assess guidelines if the types of evidence used in their development were stated as clearly as possible, for example on the front cover of published guidelines there could be a statement to the effect that "this guideline is based on a Cochrane review." The guideline appraisal project of the Health Information Research Unit at McMaster University is an example of efforts to help practitioners identify and critically evaluate clinical guidelines, and to determine their applicability to local practice.²⁵

Information from research needs to be presented in forms that are appropriate for the target audience. Guidelines from the Agency for Health Care Policy and Research have been packaged in different ways for different users; they have been packaged as a detailed report of the review with a full exposition of the evidence for researchers and decision makers, as a briefer guideline for clinicians, and as a leaflet for patients. In the United Kingdom, the Midwives Information and Resource Service has produced a series of leaflets aimed at both pregnant women and their professional carers using, when possible, evidence from Cochrane reviews.²⁶

Simply presenting research evidence to clinicians is often insufficient to ensure that it is incorporated into practice. Government directives and direct incentives such as payments can increase the speed of uptake. Sometimes powerful research findings will have an immediate effect; swift changes in practice followed the publication of research findings that sleeping position could affect mortality from the sudden infant death syndrome. ^{27 28} However, even when findings are packaged, summarised, and made relevant to clinicians further action will be needed to ensure their implementation.

A complex set of factors influences the uptake of research findings, and a variety of dissemination methods need to be used to encourage clinicians to make informed changes in their practice.²⁹ Much research on effective implementation is currently under way, but a nationally coordinated strategy to disseminate and promote important evidence from research and systematic reviews could improve implementation among healthcare professionals. National campaigns to distribute information packs, briefings, and videos of important points from research findings could speed the wider adoption of changes in practice. Such national campaigns would need to be complemented by a variety of other activities at a local level.³⁰ Local implementation strategies involving continuing education programmes, patient education programmes, and library and information outreach services could be coordinated to ensure that key research evidence is not only accessible but also acted on.

	Acknowledgments

Helpful comments were provided by Olwen Jones, Susan Mottram, Ian Watt, Trevor Sheldon, Andrew Jones, and the two referees for this paper.

Funding: None.

Conflict of interest: None.

	References

1. NHS Executive. Priorities and planning guidance for the NHS: 1997/98. Leeds: Department of Health , 1996.
2. Glasziou PP. Support for trials of promising medications through the pharmaceutical benefits scheme: a proposal for a new authority category. Med J Aust 1995; 162: 33-36[Medline].
3. Stern K. Clinical guidelines and negligence liability. In: Deighan M, Hitch S, eds. Clinical effectiveness from guidelines to cost-effective practice. Brentwood: Earlybrave Publications , 1995.
4. Doctors in the dock. Economist 1995;Aug 19:23-4.
5. URL: http://biomednet.com/.
6. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ 1994; 309: 1286-1291[Abstract/Free Full Text].
7. McKibbon KA, Wilczynski NL, Walker-Dilks CJ. How to search for and find evidence about therapy. Evidence-Based Medicine 1996; 1: 70-72.
8. Oxman AD. Checklists for review articles. BMJ 1994; 309: 648-651[Free Full Text].
9. Hayward RS, Wilson MC, Tunis SR, Bass EB, Guyatt GH. How to use clinical practice guidelines. Part A. Are the recommendations valid? JAMA 1995; 274: 570-574[Medline].
10. Wilson MC, Hayward RS, Tunis SR, Bass EB, Guyatt GH. How to use clinical practice guidelines. Part B. What are the results and will they help me in caring for my patients? JAMA 1995; 274: 1630-1632[Medline].
11. URL: http://www.nlm.nih.gov/databases/freemedl.html.
12. Kuller AB, Wessel CB, Ginn DS, Martin TP. Quality filtering of the clinical literature by librarians and physicians. Bull Med Libr Assoc 1993; 81: 38-43[Medline].
13. Marshall JG. The impact of the hospital library on clinical decision-making: the Rochester study. Bull Med Libr Assoc 1992; 80: 169-178[Medline].
14. Klein MS, Ross FV, Adams DL, Gilbert CM. Effects of on-line literature searching on length of stay and patient care costs. Acad Med 1994; 69: 489-495[Medline].
15. Urquhart CJ, Hepworth JB. The value to clinical decision making of information supplied by NHS Library and Information Services. London: British Library , 1995(British Library research and development report no. 6205.)
16. Cimpl K. Clinical medical librarianship: a review of the literature. Bull Med Libr Assoc 1985; 73: 21-28[Medline].
17. Schatz CA, Whitehead SE. "Librarian for hire": contracting a librarian's services to external departments. Bull Med Libr Assoc 1995; 83: 469-472[Medline].
18. Wallingford KT, Ruffin AB, Ginter KA, Spann ML, Johnson FE, Dutcher GA, et al. Outreach activities of the National Library of Medicine: a five-year review. Bull Med Libr Assoc 1996;84 (April suppl):1-60.
19. Rowlands JK, Forrester WH, McSean T. British Medical Association library free Medline service: survey of members taking part in an initial pilot project. Bull Med Libr Assoc 1996; 84: 116-121[Medline].
20. URL: http://wwwlib.jr2.ox.ac.uk/prise/.
21. Palmer J, Streatfield D. Good diagnosis for the twenty-first century. Libr Assoc Rec 1995; 97: 153-154.
22. URL: http://www.nlm.nih.gov/nichsr/nichsr.html.
23. Scottish intercollegiate guidelines network. Clinical guidelines: criteria for appraisal for national use. Edinburgh: SIGN , 1995.
24. NHS Executive. Improving outcomes in breast cancer: the research evidence. London: Department of Health , 1996.
25. http://hiru.mcmaster.ca/cpg/.
26. Oliver S, Rajan L, Turner H, Oakley A. A pilot study of "Informed Choice" leaflets on positions in labour and routine ultrasound. York: NHS Centre for Reviews and Dissemination , 1996.
27. Spiers PS, Guntheroth WG. Recommendations to avoid the prone sleeping position and recent statistics for sudden infant death syndrome in the United States. Arch Pediatr Adolesc Med 1994; 148: 141-146[Abstract].
28. Hilley CM, Morley CJ. Evaluation of government's campaign to reduce risk of cot death. BMJ 1994; 309: 703-704[Free Full Text].
29. Deykin D, Haines A. Promoting the use of research findings. In: Peckham M, Smith R, eds. Scientific basis for health services. London: BMJ Publishing Group , 1996.
30. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995; 274: 700-705[Abstract].