Imaging strategies for detection of urgent conditions in patients with acute abdominal pain: diagnostic accuracy study

We thank Dr. Muttalib and colleagues for their clear response in
which they discuss several potential forms of bias related to our study.
Data collection was performed prospectively using a digital web-based case
record form. The duration of complaints had a unimodal distribution, with
a mean of 1.7 days and. Indeed, we are certain that duration of complaints
is an important factor in clinical decision making.
We are aware that clinical data available during test interpretation may
affect test performance. The methods section of the manuscript states that
imaging was interpreted with clinical information. As indicated, it is
preferable to evaluate diagnostic tests in the way they are used in
clinical practice. By comparing the accuracy of the imaging strategies
after the initial clinical diagnosis we were able to assess the added
value of imaging, interpreted with clinical data. Our conclusion was that
initial ultrasound with conditional CT after the clinical diagnosis would
result in a reduction of missed urgent cases and of false-positive
diagnoses of urgent conditions.

Competing interests:
None declared

Editor – We thank Dr. Laméris and colleagues for their response.

There have been numerous articles by Whiting et al, with respect to
the QUADAS quality assessment tool for diagnostic studies worthy of
inclusion in systematic reviews, one of which Dr. Laméris cites (1).

In their response, the authors were aware of the QUADAS quality
assessment tool pointing to the known quality shortcomings of
observational diagnostic studies, and might have strengthened their study
further (2,3).

With reference to quality item 11 of QUADAS: were the reference
standard results interpreted without knowledge of the results of the index
test? We understand that the reference standard was set as an expert panel
achieving a post-hoc diagnosis (2) to enable uniformity and reduce
verification bias. With respect to non-blinding of the clinical and
imaging features (in the setting of a scientific study, not in the setting
of everyday practice), the actual contribution of imaging to diagnostic
pick-up remains unknown. The availability of clinical data during imaging
is known to affect estimates of test performance, owing to the
interpretative component of imaging - clinical review bias (4).

However, the study does fulfil quality item 12 of QUADAS: were the
same clinical data available when test results were interpreted as would
be available when the test is used in practice? ie. clinical features are
known to the radiologist in daily practice.

It should be remembered that clinicians involved in the study could
have had more hightened awareness eg. requesting imaging in cases that
they otherwise would not have – also known as context bias (5). Diagnostic
observational studies have the limitation of possible subconscious bias
about the benefits of new technology – in other study settings, a
clinician may delay making a definitive diagnosis if they know that
another test is going to be performed. The unblinded interpretation of
imaging may concentrate the viewer towards subtle features that otherwise
would have been missed, and possibly distract the viewer from other areas
of analysis (5).

This would make the diagnostic contribution of imaging with respect
to hightened clinical suspicion unknown. Without such blinding, we would
suggest that readers appreciate that the quoted sensitivity and
specificity (diagnostic performance) of imaging results are actually in
combination with clinical assessment (which would also be a function of
the clinician’s experience).

A test would be expected to perform better in severe (advanced)
disease states, as pick-up is more likely than in mild (early) pathology.
Disease prevalence is likely to be higher owing to the selection of more
severe (overt) cases at clinical referral. In this situation, severity and
prevalence can be linked, explaining apparently high sensitivity with
higher prevalence.

With respect to quality item 14 of QUADAS: were withdrawals from the
study explained? We appreciate the logistics of data collection for the
multi-centre study explaining the rather high (7.3%) proportion of
patients with incompletely recorded (and therefore selected-out) data,
although the authors’ response suggests that such cases might not have
been materially important demographically (although their clinico-
radiologic importance remains unknown).

Could the authors confirm whether data was captured manually on paper
or on electronic internet (web) based-proformas. With the latter,
incomplete data submission could have been avoided if the computer system
alerted the user to and rejected incomplete fields. Blinding could also
have been achieved with clinical / biochemical text fields collapsing down
after completion, with a flag showing the radiologist that these
assessments had actually been completed rather than omitted. This would
have enabled quality item 11 of QUADAS to be fulfilled, and once the
radiologist had completed their part of the results proforma, then the
clinical / biochemical text fields could then pop-up to enable the
unblinded “everyday practice” part to resume so that patients are not
disadvantaged by the study process in any way (enabling quality item 12 of
QUADAS also to be fulfilled). We presume that the proportion of incomplete
data would have been even higher if the research fellow did not “chase-up”
physicians to complete the data. Data export centrally for the purposes of
analysis would also have been expedited using secure internet (web)-based
electronic proformas, avoiding the need to “chase-up” prospectively
collected data.

We appreciate that in daily practice, a proportion of patients would
be discharged by emergency department physicians (ie. selected-out) before
even considering imaging. We re-iterate that within the context of a
scientific study, these patients were either very well for discharge or
had a missed diagnosis – both of which could have affected clinical
diagnostic performance figures, especially as the decision to discharge
might have been related to the inexperience of the assessing doctor. It
would have been useful to collect follow-up data for patients that re-
presented to the emergency department or were subsequently admitted; which
as the authors’ reply states was unfortunately not recorded. This would
also have strengthened the study design (6) by reducing the withdrawal
(dropout) bias from analysis (4).

Quoting Dr. Laméris’ response: “The duration of complaints and
inflammatory parameters are probably important factors for physicians when
deciding whether imaging is warranted.” As mutual clinicians, Dr. Laméris
would surely agree that this is a certainty of actual practice, not a
probability.

“If warranted, this study showed that a strategy using ultrasound
first with conditional computed tomography resulted in a high sensitivity
of urgent conditions, regardless of the duration of complaints”. This
study unfortunately did not stratify data according to the duration of
symptoms (eg. <12 hours, between 12-24 hours etc.) beyond the general
inclusion criteria of abdominal pain between 2 hours and 5 days duration.
Until this is done, it remains unknown if there is higher imaging pick-up
because patients with more advanced duration of symptoms are more likely
to have overt physical signs and more likely to manifest SIRS (systemic
inflammatory response syndrome) parameters (7) and the physician is
therefore more likely to “warrant”/request imaging (context bias,
described above) !

It would be important to show that the data for symptom duration was
evenly distributed for the patients between the protocol 2 hours and 5
days. If it is skewed towards the latter, then the abovesaid becomes a
problem. If it is skewed towards the former (2 hours) then imaging is
blessed with solving the holy-grail diagnostic challenge described in our
previous letter viz. when the history of abdominal pain has been short and
SIRS (systemic inflammatory response syndrome) parameters have not yet
manifest.

With respect to extending the role of (readily available) plain
radiography in imaging the acute abdomen, the second table described in
the authors’ response unfortunately does not appear in the printed version
of the article (3). However, we look forward to their intent to publish a
more detailed analysis of the value of plain radiographs for specific
diseases and hope that their study design might include the quality
standards described in Dr. Laméris’ response and importance of blinding
highlighted in this letter.

(1) Whiting P, Rutjes AWS, et al. The development of QUADAS: a tool
for the quality assessment of studies of diagnostic accuracy included in
systematic reviews. BMC Medical Research Methodology 2003; 3: 25.

(2) Laméris W, van Randen A, Dijkgraaf MG, Bossuyt PM, Stoker J,
Boermeester MA .Optimization of diagnostic imaging use in patients with
acute abdominal pain (OPTIMA): design and rationale. BMC Emerg Med 2007;
7: 9.

(3) Laméris W, van Randen A, van Es HW, van Heesewijk JP, van
Ramshorst B, Bouma WH, et al. on behalf of the OPTIMA study group. Imaging
strategies for detection of urgent conditions in patients with acute
abdominal pain: diagnostic accuracy study. BMJ 2009; 339: 29-33.

(4) Kelly S, Berry E, Roderick P, Harris KM, Cullingworth J,
Gathercole L, et al. The identification of bias in studies of the
diagnostic performance of imaging modalities. Br J Radiol 1997;70:1028–35.

(5) Egglin TK, Feinstein AR. Context bias: a problem in diagnostic
radiology. JAMA 1996; 276: 1752–5.

(6) Whiting P, Rutjes AW, Dinnes J, Reitsma JB, Bossuyt PM, Kleijnen
J. Development and validation of methods for assessing the quality of
diagnostic accuracy studies. Health Technology Assessment 2004; Vol. 8:
No. 25.

(7) American College of Chest Physicians / Society of Critical Care
Medicine Consensus Conference: definitions for sepsis and organ failure
and guidelines for the use of innovative therapies in sepsis. Crit. Care
Med. 1992; 20(6): 864-74.

Competing interests:
None declared

Editor – The issues raised in the correspondence generated from Dr.
Laméris et al. paper (1) include the experience of the clinical and
radiological assessors and access to imaging facilities. These
characteristics may indeed differ in other countries in everyday practice
around the globe.

In the methods section, ultrasonography and computed tomography (CT)
were read blinded to each other. For the purposes of a research study,
imaging should also have been blinded to the clinical features; otherwise
the sensitivity and specificity of imaging would actually have been in
combination with clinical assessment. Can the authors confirm that there
was blinding in this respect?

In the results section, data collection was not completed for 80/1101
patients, equating to 7.3%. These patients were therefore selected-out of
the study but may have been relevant. Can the authors give reasons for
this high figure?

Patients discharged from the emergency department without imaging
were not included in the study ie. also selected-out. These patients were
either very well for discharge or had a missed diagnosis – both of which
could have affected clinical diagnostic performance figures. Can the
authors give any data regarding the proportion of patients that re-
presented to the emergency department or were subsequently admitted?

It remains important not to embrace imaging as a complete substitute
for clinical assessment. As pointed out in the accompanying editorial (2),
ultrasound is well-known to be operator dependent and we would add that
computed tomography (CT) is known to be interpreter dependent; especially
when not proof-read. There is much to applaud with making a working
clinical diagnosis and utilising investigations eg. haematology,
biochemistry and imaging to confirm / refute the diagnosis on a background
of clinical re-evaluation.

Various scoring and assessment systems for the acute abdomen are
indeed surrogates for recognising the unwell patient ie. the sepsis arm of
SIRS - the systemic inflammatory response syndrome (3) viz. two or more of
the following parameters:

Temperature <36 or >38 Celsius

Tachycardia >90 bpm

Tachypnoea >20

White cell count <4 or >12 (x10^9 cells/L)

With the exception of raised amylase in acute pancreatitis and
hallmarks of bacteruria in pyelonephritis, patients with the above
features and physical signs of an acute abdomen invariably proceed to
surgery as imaging would not alter clinical management (if anything, there
would be a delay to surgery). However, imaging would be expected to reduce
false positive clinical diagnosis especially when the history of abdominal
pain has been short and SIRS parameters have not yet manifest. This is
where the significant diagnostic challenge rests. Can the authors give any
figures or stratify their data according to the duration of symptoms (eg.
<12 hours, between 12-24 hours etc.) beyond stating blanket inclusion
criteria of abdominal pain between 2 hours and 5 days duration? We are not
aware of any imaging studies presented in this more meaningful way to-
date.

There is much to be said about extending the role of (readily
available) plain radiography in imaging the acute abdomen, which was not
considered in the present study. Although sonographic detection of
pneumoperitoneum (4) has been described, plain radiography in the left
lateral decubitus position has been recommended as an adjunct to the erect
chest radiograph (5) and can detect as little as 1 ml of air without the
use of CT (6). With respect to plain abdominal radiography, the Rigler
sign is well-known to indicate [large] pneumoperitoneum (7) often colonic
in origin. Gastrografin transit with serial abdominal radiography not only
demonstrates adhesive small bowel obstruction, but can also predict the
likelihood of resolution with conservative management. There is good
evidence that the use of water-soluble contrast reduces the length of
hospital stay in this situation (8).

The present study did not include pregnant women, in view of
radiation exposure from CT (although such patients might have usefully
been included in a non-CT arm of the study). Difficulty in clinical
diagnosis of abdominal pain in this group remains challenging. The
position of inflamed viscera (eg. appendix) may alter with the rise of the
gravid uterus and borderline / mildly elevated white cell counts can be
expected in pregnancy. Graded compression sonography has been described in
this situation (9).

(1) Laméris W, van Randen A, van Es HW, van Heesewijk JPM, van
Ramshorst B, Bouma WH, et al. on behalf of the OPTIMA study group. Imaging
strategies for detection of urgent conditions in patients with acute
abdominal pain: diagnostic accuracy study. BMJ 2009; 339: 29-33.

(2) Dixon AK, Watson CJ. Imaging in patients with acute abdominal
pain: Emerging evidence points to a new sequence of investigations. BMJ
2009; 339: 1-2.

(3) American College of Chest Physicians / Society of Critical Care
Medicine Consensus Conference: definitions for sepsis and organ failure
and guidelines for the use of innovative therapies in sepsis. Crit. Care
Med. 1992; 20(6): 864-74.

(4) Lee DH, Lim JH, Ko YT, Yoon Y. Sonographic detection of
pneumoperitoneum in patients with acute abdomen. AJR 1990; 154: 107-9.

(5) Gaines WG. Pneumoperitoneum in perforated peptic ulcer: factors
in Roentgenographic demonstration. California Medicine 1953; 78(6): 508-
512.

(6) Miller RE, Nelson SW. The roentgenologic demonstration of tiny
amounts of free intraperitoneal gas: experimental and clinical studies.
AJR 1971; 112: 574-585.

(7) Lewicki AM. The Rigler Sign and Leo G. Rigler. Radiology 2004;
233(1): 7-12.

(8) Abbas S, Bissett IP, Parry BR. Oral water soluble contrast for
the management of adhesive small bowel obstruction. Cochrane Database Syst
Rev. 2007; 18(3): CD004651.

(9) Lim HK, Bae SH, Seo GS. Diagnosis of acute appendicitis in
pregnant women: value of sonography. AJR 1992; 159: 539-42.

Competing interests:
None declared

"Surgical residents evaluated 74% (n=757) of patients, and emergency
medicine residents evaluated the other 26%...The mean clinical experience
of the residents was 25 months (range 2 months to 8.7 years)". What is the
practice in Holland? Have surgical residents done six months surgery as a
houseman, worked six months as a casulaty officer/ER physician and written
their primary fellowship before begining their residency? What of the
emergency medicine residents? Have they completed six months as a
surgivcal house officer?

In the US emergency physcians never get the opportunity to do a
surgical job and medical students are given almost zero opportunity to
scrub and learn some of the basics of surgery. Furthermore surgical
residents have not done either an HS or an HP job before entering their
residency programs. Even then they are heavily occupied doing "scut work"
and rarely get the opportunity to get into the OR until their second and
thord years. Additionally they are not required to write a primary FRCS
and do not write their boards until they have completed their residencies.

The inexperience of those evaluating acute abdomens in the ER today
may have a lot to do with the rise in imaging. Finally the objectives of
an emergency physician and radiologist are different from those of a
surgeon, the former being intent upon making an accurate diagnosis and the
latter intent upion deciding whether there is a need to operate or not.
For a surgeon making an accurate diagnosis is more related to what incsion
to use. That decsion can be deferred until after the unduction of
anaesthesia and reexamining the relaxed abdomen. The opposing objectives
clearly influence the perceived need for imaging.

Competing interests:
None declared