Introduction

Top Abstract Introduction Methods Results Discussion References

Doctors are urged to practise evidence based medicine when faced with questions about how to care for their patients.¹ They are advised to ask questions that can be answered with evidence and to evaluate the results of original research.^1-3 But this advice may be difficult to follow in the pressurised atmosphere of a busy practice.^4-6 Doctors are overwhelmed by the amount of information available, yet they often cannot answer their questions about specific clinical problems. ^{5 7-9}

We aimed to describe the range of obstacles that occur when trying to obtain evidence based answers to real clinical questions, and to build a taxonomy that characterises the problems that arise when searchers attempt to answer doctors' questions. Doctors need up to date, high quality answers at the point of care within minutes.⁵ Before these objectives can be met with new information systems, the problems with current resources and search strategies need to be described.

	Methods

Top Abstract Introduction Methods Results Discussion References

Selection of questions
We collected 1101 questions from 103 family doctors in Iowa. Briefly, after each consultation an observer asked the doctor to report any questions that occurred about how to care for the patient. We collected straightforward questions ("What is the dose of metformin?") as well as vague uncertainties that would normally be kept to oneself ("I'm not sure what this rash is, but I'm going to call it a contact dermatitis for now."). From these 1101 questions we selected a random sample of 200 questions. Through an iterative process of reviewing questions, creating a classification scheme, coding questions, and revising the classification scheme, we developed a method of identifying questions that were potentially answerable with evidence. This led to the development of an "evidence taxonomy" (box). ^{10 11} Using this taxonomy, we found that 106 questions (53% of the original 200) could potentially be answered with evidence.

We agreed on three criteria for selecting the two questions to be answered by all investigators: the question should be clearly stated, there should be a high likelihood of finding good quality evidence to answer it, and the answer should potentially have an impact on patient care. By using these criteria we selected "What is the proper treatment of gastro-oesophageal reflux disease (GERD)?" and "What should I use for atopic dermatitis?"

Answering questions
We did not follow a standardised search strategy because we wanted to study obstacles related to the strategy. We searched textbooks, journal articles, and various computer applications, but we did not seek individual consultations with humans. Working independently, the investigators completed searches that they thought were sufficient to avoid missing important evidence. While searching, the investigators used a modified "think aloud" method to write field notes that documented the obstacles they encountered. ^{12 13}

Development of the taxonomy
We used three data sources to develop the initial taxonomy. The primary source consisted of obstacles documented in field notes written by the investigators as they attempted to answer the questions. The second source comprised frustrations that the investigators had encountered while answering other clinical questions. The third source consisted of problems reported in the literature. ^{1 14-16} The obstacles were described and organised into a taxonomy by using qualitative text analysis. The taxonomy was developed with an approach in which initial "codes" (obstacles described in the "think aloud" field notes) were augmented with obstacles described in the literature and previously encountered by the investigators.¹⁷

Validation of the taxonomy
To help validate the taxonomy, we first asked four volunteers (two medical librarians and two university family doctors) to answer four additional questions from the same dataset. Each volunteer coded their own field notes and identified obstacles that were not optimally characterised in the existing taxonomy. Secondly, we asked 21 practising doctors (purposively selected from a list of former trainees from practices in Iowa) to describe on paper the problems they encountered when attempting to answer one of their own questions. Thirdly, we completed 16 half day observation periods involving four randomly selected practising doctors in Iowa (four observation periods per doctor). We asked these doctors to "think aloud" as they attempted to answer their own questions. Based on these three additional sources of data, we added four obstacles to the taxonomy. The final version of the taxonomy was approved by all investigators.

	Results

Top Abstract Introduction Methods Results Discussion References

The box shows examples from the taxonomy of obstacles, with descriptions of each obstacle (a full list of the taxonomy is available on bmj.com). The taxonomy was organised according to the steps in asking and answering questions^18-20: recognise a gap in knowledge, formulate a question, search for relevant information, formulate an answer, and use the answer to direct patient care. Most of the obstacles were supported by the data we obtained, but a few were primarily generated from the previous experiences of the investigators or from the literature.

Several obstacles seemed particularly salient because they recurred in the various procedures for data collection, and they were characterised as fundamental problems by the investigators and practising doctors. These were the excessive time required to find information, difficulty modifying the original question, which was often vague and open to interpretation, difficulty selecting an optimal search strategy, failure of a seemingly appropriate resource to cover the topic, uncertainty about how to know when all the relevant evidence has been found so that the search can stop, and inadequate synthesis of multiple bits of evidence into a clinically useful statement.

The obstacles related to evidence fell into two main categories. Firstly, the available evidence was inadequate to directly answer the question either because studies had not addressed the question ("Is smoking a risk factor for sinusitis?") or because the studies that had addressed the question provided incomplete information. For example, when answering the question about treating gastro-oesophageal reflux disease, we found rigorous comparisons between lansoprazole and placebo and between omeprazole and placebo but the comparisons between lansoprazole and omeprazole were less definitive.^21-23 Secondly, even when the evidence was adequate to answer the question, further obstacles hindered its use in the clinical setting. Available evidence often consisted of individual study results, which had not been synthesised or interpreted for clinicians. The following field notes were written by one of the investigators as he attempted to answer the question, "What should I use for atopic dermatitis?"

Therapies include: ciclosporinpossibly effective; borage oilprobably not effective; UVA1works; primrose oil with water and oil emulsionprobably effective; topical doxepinprobably not effective; mite eliminationlikely to be effective; topical cromolynlikely to be effective; topical tacrolimusprobably effective; SEZ ASM 981possibly effective.

Each investigator spent a median of 95 minutes (range 13 to 639 minutes) answering the question about gastro-oesophageal reflux and 45 minutes (range 17 to 374 minutes) answering the question about atopic dermatitis.

The final version of the taxonomy comprised 59 obstacles. The four volunteer coders used 35 problems to code their field notes and made four suggestions to improve the taxonomy. For example, both librarians noted their lack of medical training as an obstacle to formulating an answer. Ten of the 21 practising doctors responded to our request to describe obstacles that arose as they answered one of their own questions. All of the obstacles reported by these doctors had been described in the existing taxonomy. Also, we collected 96 questions during 16 office observations from four additional practising doctors. These data led to the addition of four obstacles to the taxonomy: failure to initiate a search due to doubt about the existence of relevant information, ready availability of consultation, which leads to a referral rather than a search, uncertainty about the meaning of null search results, and resource not clinically oriented.

	Discussion

Top Abstract Introduction Methods Results Discussion References

Obstacles arise when searching for evidence based answers to doctors' questions: we identified 59. Among the most salient were inadequate time to search for information, failure of the resource to address the topic, and inadequate synthesis of multiple bits of evidence into a clinically useful statement. Practising doctors often decided not to pursue their questions because they doubted the existence of useful information in available resources.

Implications
After quantifying and prioritising the obstacles we found, the taxonomy we developed could be used to write recommendations for authors as they attempt to produce clinically useful material. Authors will be most effective if they anticipate the needs of busy clinicians who often have only a minute or two to find information.⁵ For example, authors who name the drug of choice for a specific condition could include essential prescribing information (dosage, drug interactions, safety in pregnancy), so that the clinician does not waste time consulting another resource. Clinically oriented resources could be written in a question and answer style rather than a disease and topic style. The ongoing surveillance of doctors' changing questions could help keep resources current. Questions without adequate answers could help guide research and funding priorities. Until such research is completed, such questions may prompt the use of holistic clinical care and other alternatives. We often found it helpful to modify questions from the way they were originally stated by the doctor. Such modifications could be developed into recommendations for doctors, as they formulate their questions, and for intermediary searchers, who may play a larger part in the future, as they help doctors practise the best medicine.²⁴

Conclusions
To meet the needs for clinical information, doctors must be aware of their gaps in knowledge and then formulate questions that can be addressed by available resources or patient specific consultations. When faced with a gap in knowledge, doctors must decide whether to do the best they can with their current knowledge or to expand that knowledge by formulating and answering a question. Practising doctors do not have time to search multiple sites or scroll through long text. Nor do they have time to search multiple textbooks or perform literature searches for most of their questions. They need to pick the right resource the first time, the information in that resource needs to be readily found, and all the information must be there. Although it remains to be shown, we believe that systems designed to overcome the obstacles we identified will improve the asking and answering of questions and potentially patient outcomes.