Just in time information for clinicians: a questionnaire evaluation of the ATTRACT project
BMJ 2001; 322 doi: https://doi.org/10.1136/bmj.322.7285.529 (Published 03 March 2001) Cite this as: BMJ 2001;322:529
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Jon Brassey’s evaluation (1) of his ATTRACT project (provision of
evidence based information to GPs in Gwent in less than 6 hours) deserves
further comment. The main evaluation tool was an anonymised questionnaire
sent to 50 General Practitioners. The format of the questionnaire was not
published, thus preventing readers from making a judgement regarding its
validity and reliability. Also, Gwent has over 300 GPs which puts the 42
respondents into some context. What were the opinions of the 250 GPs who
were not surveyed?
Unfortunately, only scant details were available regarding Brassey’s
search strategies. The main sources of information included the Cochrane
Library, the “Trip” Database, Pubmed and the internet. It seems strange
that Embase has been ignored because this would have added an extra 1000
journals and also given a more European flavour to the data. Although the
internet contains valuable information, much of its content is valueless
and may even be dangerous (2) - it is important to state how the quality
of these data was assured. In addition, complete reliance on electronic
databases has been shown to be ill advised. In one study, fewer than half
of the references for a systematic review were identified using electronic
databases – 24% of references would have been missed entirely had the
authors not contacted experts in the field (3) .
Brassey’s assertion that these reviews are conducted rapidly, “within
6 hours”, is worrying. The implication is that his reviews are written
using available abstracts rather than full references. Pitkin found that
up to 68% of abstracts were deficient, containing information which was
inconsistent with or completely missing from the body of the text (4).
Presumably, Brassey ignored those citations which contained no abstract at
all.
In fairness, we have previously used a rapid search technique for
identification of adverse drug reactions although the focus of our
research was to detect mere signals of potential effects (5). However,
the ATTRACT method is being used to answer complex clinical queries.
There is a genuine concern that this rapid technique will result in an
over-simplification of the issues, biased answers or simply wrong
information.
Finally, Brassey implies that the study was an appropriate evaluation
of his service. This is disturbing because none of his queries appears to
have been properly and independently evaluated. In order to do this, one
would have expected a sample of his queries to have been compared against
a gold standard i.e. staff trained in searching and appraisal of
literature taking their time to evaluate each query individually. The
most puzzling aspect of all is how an esteemed journal such as the BMJ has
(apparently) allowed a paper of such low scientific merit to be published
in the first place.
Yours faithfully,
J. P. Hampson MSc; MPH; MIInfSc
Teacher Practitioner
Gwenfro Academic Unit,
Wrexham Maelor Hospital,
Croesnewydd Road,
Wrexham
LL13 7TD
References
1. Brassey J, Elwyn G, Price C, Kinnersley P. Just in time
information for clinicians: A questionnaire evaluation of the ATTRACT
project. BMJ 2001; 322: 529-30
2. Chalmers I. Invalid Health Information is potentially lethal. BMJ
2001; 322:998
3. McManus RJ, Wilson S, Delaney BC, Fitzmaurice DA, Hyde CJ, Tobias
RS, Jowett S, Hobbs FDR. Review of the usefulness of contacting other
experts when conducting a literature search for systematic reviews. BMJ
1998; 317: 1562-3
4. Pitkin R. M., Brannigan M. A., Burmeister L. F. Accuracy of data
in abstracts of published research articles. JAMA 1999; 281:1110-11
5. Hampson JP, Harvey JN. A systematic review of drug-induced ocular
reactions in diabetes. Br J Ophthalmol 2000; 84: 144-9
Competing interests: No competing interests
Dear Sir
We read with interest the article by Brassey et al (1) which
described the “ATTRACT project” in Wales, which received 193 queries from
GPs in a 13-month period. Many of these involved therapeutics. However,
we regret his apparent lack of awareness of the UK Medicines Information
Service.
Medicines Information (MI), formerly drug information, began in 1966.
A centre exists in most general hospitals in the UK, receiving enquiries
from healthcare professionals in both primary and secondary care on all
aspects of drug therapy. Centres are staffed by experienced pharmacists
whose skills include linking information provision with clinical
interpretation. These pharmacists use a variety of evidence-based,
critically assessed resources to provide the information needed to answer
enquiries received. In 1997 these centres together answered more than
250,000 enquiries (2) (approx. 1,000 per local centre), all of which were
concerned with drug therapy. The network of centres provides a locally-
based and responsive service, with co-ordination at a UK level. The Chief
Pharmacists of the four home countries have recently acknowledged the
expertise, impartiality, responsiveness and efficiency of MI pharmacists
through a national strategy which they have commended to healthcare
providers (3).
The co-ordination of MI services throughout the UK has facilitated
many developments. We have established centres of national expertise with
specialist knowledge in specific areas of therapeutics such as the use of
drugs in pregnancy and complementary medicine. Nationally we conduct
detailed reviews of drugs both pre- and post- launch. We are also
developing a comprehensive website.
There are nationally accepted standards for the provision of MI
services and individual centres are regularly audited externally against
these (4). Those who use an information service and those who fund it need
to be assured of its quality. MI services can provide this assurance.
Our history has been one of collaboration and networking at all
levels: Medicines Information works closely with the National Prescribing
Centre, NICE and NHS Direct. Locally, we work with healthcare
professionals from both primary and secondary care. Many interface
prescribing committees, health authorities, PCTs, LHGs and NHS librarians
also benefit from a close liaison with their local medicines information
pharmacist. Collaboration reduces duplication of effort and consequently
makes more effective use of limited resources.
We would welcome the opportunity to share our experience with
ATTRACT, and hope that this will foster a spirit of collaboration.
Yours sincerely,
Mrs Fiona Woods
Director
Welsh Medicines Information Centre,
University Hospital of Wales, Heath Park, Cardiff
E-mail : fiona.woods@cardiffandvale.wales.nhs.uk
Mr Peter Golightly
Director
Trent Regional Medicines Information Centre,
Leicester Royal Infirmary,
Leicester LE1 5WW
Mrs Anne Lee
Principal Pharmacist
Area Medicines Information Centre,
Glasgow Royal Infirmary,
84 Castle Street,
Glasgow G4 0SF
Dr Simon Wills
Head of Wessex Drug & Medicines Information Centre,
Mailpoint 40,
Southampton General Hospital,
Southampton SO16 6YD
References
1) Brassey J, Elwyn G, Price C and Kinnersley P
Just in time information for clinicians: a questionnaire evaluation of the
ATTRACT project.
British Medical Journal 2001; 322:529-530 (3 March)
2) Hands D, Judd A, Golightly P and Grant E.
Drug information and advisory services – past, present and future.
Pharmaceutical Journal 1999: 262:160-162
3) Better Information for Managing Medicines : A strategy for
pharmacy's Medicines Information Service in the NHS UKMi April 2000
4) Golightly P, Grant E, McKee C, Simister K and Woods F.
Made to measure.
Health Service Journal 1994; 104(5414):22
Competing interests: No competing interests
Editor,
We support this work by Brassey (1) and write to share an experience
of adopting a similar approach to policy making.
Locally, our approach (2) has been similar, though not identical, and
this has led us to consider the relative merits of the two projects. A key
difference is that whereas the ATTRACT project focuses on the clinical
questions generated by GPs, our local Health Authority's Effectiveness
Facilitation Unit serves a wider clientele who generate questions around
Health Policy.
We have adopted a similar method of summarising evidence, which we
have termed a 'Systematic Overview'. We attempt to answer questions of
effectiveness or harm by arriving quickly at a view of what is 'accepted
best wisdom'. In doing this, we look to reliable databases of pre-
appraised material, whose methodological filter is robust e.g. DARE. We
also go to authoritative, usually online, sources of National guidance and
sometimes to recognised experts in the field. This latter course is
usually only necessary when there are few or no 'off the shelf' reviews of
evidence. Our aim is to produce a well-considered, balanced and pragmatic
report within about 2 weeks, which is slower than the turnaround achieved
by ATTRACT. Nevertheless we feel that this is 'just in time' for our
audience, for two main reasons.
First policy decisions, on behalf of a population, do not have the
immediacy of clinical decisions, on behalf of an individual. Also the
extensive evidence base that often surrounds clinical decision making is
more likely to be lacking, and appraisal/evaluation tools less well-
developed hence the need for more time to search, appraise and summarise.
Second, because of the nature of the client (often not medically
qualified and sometimes from outside the NHS), an additional step in the
dissemination/quality control process is sometimes needed. Where the
client is not a health professional, reports are disseminated by
professionals with skills in communicating information to the public,
usually a consultant in public health medicine or communicable disease
control. Nevertheless, the approach still retains the advantages
identified by Brassey et al i.e. the most cost-effective use of staff
resources by using people with specific training and experience in
undertaking reviews of evidence.
It is clear that the 'just in time' approach of Brassey et al is
appropriate and valued by their clients. We have not yet evaluated our
service but will now consider doing so.
Helen Thornton-Jones (a,b)
Senior Lecturer
Susan Hampshaw (b)
Research Associate
Hora Soltani (a)
Research Officer
Andrew Taylor (a)
Health Economist
(a) Effectiveness Facilitation Unit, Health Policy and Public Health
Department, East Riding and Hull Health Authority, Health House, Willerby
HU10 6DT
(b) Public Health Research Unit, The University of Hull, East Riding
Campus, Coniston House, Willerby HU10 6NS.
(1) Brassey J, Elwyn G, Price C and Kinnersley K. Just in time
information for clinicians: a questionnaire evaluation of the ATTRACT
project. BMJ 2001;322:529-530
(2) Madhok R, Allison T, Kingdom A and Ross D. Clinical Governance:
experience from a health district. Journal of Clinical Excellence 2 2000;
2:139-146
Competing interests: No competing interests
I am aware of the various hospital-based information
services. The very fact that ATTRACT is used so much, and
valued so highly by the GPs in Gwent speaks volumes.
The key issue is giving information in formats that
clincians can use. What we were asked to supply, when we
asked the GPs, was for a service that was quick and produced
summaries of the evidence. We delivered with a turnaround
time of 6 hours and the evidence summarised onto less than
one side of A4 using sources of the highest quality.
When the hospital-based information services can match, or
even come close, to this sort of service I would agree that
the need for ATTRACT is questionable - but this is some way
off. In the meantime let the clinicians decide.
Competing interests: No competing interests
The ATTRACT project in General Practice could well be extended to
individual specialties. It appears to be an elegant way of coping with
information overload.
However I note that the scheme has not been continued. It would be helpful
to know why this is and to know the ongoing costs and pitfalls in
maintaining such a service which in Dermatology would need to be
nationwide.
Competing interests: No competing interests
The well-established UK Medicines Information network already
provides a query answering service to clinicians, nurses, other health-
care workers and patients. Although it is based in hospital pharmacy
departments, general practitioners and others in primary care use the
service. A wide range of paper, electronic and external sources are
utilised to provide answers as rapidly or as in depth as requested by the
enquirer.
The need for ATTRACT over and above the Medicines Information service
is not clear.
Competing interests: No competing interests
I used the service and found it readily accessible and user-friendly.
It was equally useful to be able to scan other doctors enquiries. It
served me as an on-line PUN's and DEN's, and overcame some of the
professional isolation of general peactice. It would be interesting to
look at the cost of each enquiry dealt with, but to bear in mind that the
number of people subsequently accessing the data add to the justification
for the cost incurred.
I hope the service will be resumed soon, why indeed did it stop?
Competing interests: No competing interests
In defence of ATTRACT
John Hampson's criticism of the ATTRACT process can be divided into
two areas. Firstly, why the BMJ published the paper as it was of such low
scientific merit? I cannot answer on behalf of the BMJ, that is for them
to defend. However, from my perspective ATTRACT is a major breakthrough
in providing 'Just in time information' for clinicians - isn't that a
justification?
Moving onto the second major criticism - the actual ATTRACT process.
A lot of his concerns are due to the much shortened entry that the BMJ
finally accepted (600 words compared to the original 1,400 submitted). If
this had been accepted, issues such as "What were the opinions of the
remaining 250 GPs who were not surveyed?", "Unfortunately, only scant
details were available regarding Brassey's search strategy.." would have
been answered
Moving onto to specific criticisms:
Embase was not used for a wide variety of reasons. The main one
being that the whole point of the ATTRACT process is that it is rapid and
therefore compromises have to be made. Again the purpose of the service
is not to conduct a systematic review for each question but to do better
than an average GP would. Embase is a useful tool, but so are others: we
basically had to have a cut-off point in order to answer questions within
a clinically useful timescale. Again this need for speed answers why we
rely on electronic resources.
We are aware of the deficiencies of using the Internet and only use
it as a last resort - and even then we are careful with the information
obtained. I would say that in less than 10% of our answers we use the
Internet and even then it is often for background information.
As for the accuracy of abstracts, I don't feel that the Pitkin
article has much bearing on the ATTRACT process. In the article, Pitkin
states "Many of the discrepancies identified were quite minor and not
likely to cause serious misinterpretation". In fact, after an exchange of
e-mails with Pitkin, he states "The frequency of errors severe enough to
lead to an erroneous conclusion is very low. I do not recall any cases in
which the actual conclusion was stated incorrectly" and "I think what you
are doing seems fine and appropriate for the clinical situation. I was in
clinical practice for many years and I understand fully the urgency of
getting the best answer".
As for the accuracy of our answers, we have conducted numerous
evaluations using a variety of methodologies. In all of these we have
produced results in line with 'gold standard' resources. We are currently
embarking on a large scale evaluation, which hopefully the BMJ will
publish within the next two years.
I think the most pertinent aspect of ATTRACT was stated by Pitkin "…I
understand fully the urgency of getting the best answer". It is very easy
to be critical but what are the alternatives? I can only assume that if
Hampson is a clinician then he is highly atypical. If a doctor has a
patient in front of them and has a knowledge gap (which occurs with very
high frequency) what is s/he supposed to do? A few suggestions: conduct a
systematic review (cost £20,000 and take 6 months); read a single article
from a journal (surely more biased than looking at a breadth of papers);
believe the drug rep (!); ask a peer (highly likely but is it
acceptable?); conduct a search (frequently poorly), order the papers,
conduct a full critical appraisal etc.
The bottom line is that ATTRACT is a pragmatic approach to freeing up
the research literature and we feel we do it rather well.
Competing interests: No competing interests