The accuracy of clinical symptoms and signs for the diagnosis of serious bacterial infection in young febrile children: prospective cohort study of 15 781 febrile illnesses
BMJ 2010; 340 doi: https://doi.org/10.1136/bmj.c1594 (Published 20 April 2010) Cite this as: BMJ 2010;340:c1594
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
We appreciate the group's ability to conduct and describe the study
clearly and thoroughly so the readers can follow the developed work.
However we would like to highlight some points that could be rethought so
that work can get even better.
First of all, the inclusion criterion that selects children whose parents
report that they "felt hot" can cause a selection bias, because parents do
not have the clinical ability to evaluate the body temperature of their
children and also they have the tendency to overvalue the signs and
symptoms, in order to seek better care for children.
A second point is that despite the fact that cases of Osteomyelitis, Septic
Arthritis, and Meningitis are rare events, we consider important their
presence in the analysis so that we can size up their impact on serious
bacterial infections.
Finally, being faithful to the definitions of diagnosis of urinary tract
infection, pneumonia, and bacteraemia in serious bacterial infections, we
believe is not suitable to include in the analysis patients who had
radiological and microbiological tests for the infections described above.
We would like to congratulate you for the great work and the great
importance that this study shows, as febrile illnesses are quite
common causes of children seeking medical assessment.
Competing interests:
None declared
Competing interests: No competing interests
We read with great interest the paper of Craig et al. on the accuracy
of clinical symptoms to diagnose serious infections in febrile children
(1). The authors identified important clinical indicators of urinary tract
infections, pneumonia and bacteraemia. Data were used from febrile
children attending the emergency department, with exclusion of those being
evaluated in another hospital initially. In addition children with cancer
and transplant recipients were excluded because of affected immune
function and different threshold for treatment and disease frequency. We
question the generalisability of the results of the study by including
other children with an increased risk of serious bacterial infections or
complicated course due to underlying illnesses such as pulmonary, cardiac,
renal, or neurologic disease and immunodeficiency.
In clinical practice the predictors identified by Craig et al. will
be used to decrease or increase the prior risk above or below acceptable
thresholds to refrain or to start treatment or to perform additional tests
(2). In a sample of 1100 attendencees at our pediatric emergency
department, Erasmus MC – Sophia, Rotterdam in 2008 (3)we observed a
presence of 22% serious bacterial infections (SBI) in children with
underlying illnesses as described above, compared to 12% in previously
healthy ones. Given this higher prior probability on SBI in this
particular subgroup, the change from prior to posterior probability of SBI
by the presence or absence of identified predictors by Craig et al. may be
too limited to affect the clinicians’ management (impact). For example, a
5% reduction of the prior risk in children with comorbidity to 17% may be
less informative than the same reduction in children without comorbidity.
However, definitions of decision thresholds to initiate or to refrain from
further steps in management of febrile children are lacking.
More important, the distribution of SBI type in febrile children with
underlying diseases may be different from previously healthy ones. Due to
a different outcome distribution, there may be a difference in predictive
value in children with underlying diseases compared to previously healthy
ones (difference in case-mix) (4). Consequently, the combined set of
predictors as a whole may over- or underestimate the probability of SBI.
Moreover, some predictors may contribute more or less to the diagnosis in
children with comorbidity compared to those without. Therefore, validation
studies on children with underlying illness are necessary to give insight
in the applicability of the results and the need for adjustments to
specific populations.
References
1. Craig JC, Williams GJ, Jones M, Codarini M, Macaskill P, Hayen A, et
al. The accuracy of clinical symptoms and signs for the diagnosis of
serious bacterial infection in young febrile children: prospective cohort
study of 15 781 febrile illnesses. BMJ. 2010;340:c1594.
2. Oostenbrink R, Moons KG, Bleeker SE, Moll HA, Grobbee DE. Diagnostic
research on routine care data: prospects and problems. J Clin Epidemiol.
2003 Jun;56:501-6.
3. van Veen M, Steyerberg EW, Ruige M, van Meurs AH, Roukema J, van der
Lei J, et al. Manchester triage system in paediatric emergency care:
prospective observational study. Bmj. 2008;337:a1501.
4. Steyerberg EW. Chapter 19 in Clinical prediction models. A practical
approach to development, validation, and updating. New York: Springer;
2009.
Competing interests:
None declared
Competing interests: No competing interests
Dr Craig and colleagues propose a diagnostic prediction model to
improve clinical decision making in children with fever suspected of
having a serious bacterial infection (1). The model aims to discriminate
between the most common serious bacterial infections (urinary tract
infection, pneumonia, and bacteraemia) rather than considering them as a
single category of 'serious bacterial infections'. We encourage this
multinomial approach, since the diagnostic work up and treatment for each
type of infection is different; the statistical analysis hence matches
well with clinical practice. However, new methodological problems are
introduced. Here, we like to highlight one problem in particular: the
temptation to select patients on the outcome.
The authors identified six different types of serious bacterial
infections. However, the model was developed excluding patients with one
of the three less common infections (osteomyelitis, septic arthritis or
meningitis). The authors argue that 'the risk of unreliable and misleading
models was unacceptably high, given the very low frequency of the
outcomes'. Although the argument may seem correct, patient selection on
the outcome is incompatible with the aim of the study, which is to
estimate absolute risks of infections. The study population should
therefore consist of consecutive patients suspected of a serious bacterial
infection (2,3). This means that the outcome (the type of infection) is
unknown at the time of patient selection. Exclusion of patients with a
particular outcome results in biased risk estimates for the other serious
bacterial infections. The effect of patient selection is minor in the
current research given the small number of excluded patients (n=26).
Nevertheless, the authors could have mentioned this methodological
pitfall.
Alternatively, patients from a small outcome category might have been
classified in the outcome category with similar consequences. For
instance, patients with meningitis have the same diagnostic work up as
patients with bacteraemia and could have been combined in one outcome
category. Indeed, the predictive effect for typical signs of one of the
diagnoses, such as neck stiffness for meningitis, will be diluted by the
patients with the other diagnosis (bacteraemia). Unfortunately, this is
inherent to the diagnostic problem. We like to mention one possible
exception. Introduction of new routine childhood vaccination for a
particular infection may eliminate that infection; the outcome category
will not occur anymore in the future and patients having this infection
may be excluded from the analysis. The reasoning remains dangerous, since
complete elimination of infections is exceptional. We emphasize that
further evaluation of the proposed diagnostic model in new patients from
different centres is warranted.
1. Craig JC, Williams GJ, Jones M, Codarini M, Macaskill P, Hayen A,
et al. The accuracy of clinical symptoms and signs for the diagnosis of
serious bacterial infection in young febrile children: prospective cohort
study of 15 781 febrile illnesses. BMJ 2010;340:c1594.
2. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig
LM, et al. The STARD statement for reporting studies of diagnostic
accuracy: explanation and elaboration. Clin Chem 2003;49(1):7-18.
3. Moons KG, Biesheuvel CJ, Grobbee DE. Test research versus
diagnostic research. Clin Chem 2004;50(3):473-6.
Competing interests:
None declared
Competing interests: No competing interests
Addressing selection bias and generalisability
Vergouwe and colleagues (and also raised by Cauduro) suggested the
exclusion of 26 children who had osteomyelitis, septic arthritis, or
meningitis resulted in biased risk estimates for other serious bacterial
infections, and suggested we should have combined those with meningitis
with the group with bacteraemia. Given the sample size (n= 15,781), and
the number of episodes of serious bacterial infection (n = 1140), we agree
with Vergouwe that the effect of exclusion of 26 children is “minor” at
worst, and effectively negligible. We disagree with the alternative
approach proposed. This would have resulted in biased estimates for a
combined category of meningitis and bacteraemia, two conditions with very
different diagnostic and treatment pathways, which was the basis of the
multinomial approach we used, and paradoxically supported by Vergouwe.
Oostenbrink suggests that our data may not be transferable to children
with comorbidities because of their different prior probabilities and
different post test treatment thresholds used by clinicians for this
patient group. We would suggest that the inclusion of these children
increased not decreased the generalisability of our study. We found that
this characteristic, underlying chronic disease, did not contribute any
additional diagnostic information for the detection of serious bacterial
infection, over and above routinely collected data on signs and symptoms.
We agree that clinicians do adjust their treatment thresholds and so would
be more likely to treat children with comorbidity than those without
comorbidity at a given probability of disease because of concerns about
the severity and prognosis of infection in children with underlying
disease but these considerations occur once the diagnosis is made and is
therefore beyond the scope of our study.
In reply to Cauduro, parental report of fever was not a source of
selection bias given the inclusion criteria were effectively applied
equally to the diagnostic model and physician estimates. It is also a
frequent reason why children present to physicians for assessment and our
aim was to provide a diagnostic model to support these physicians in their
diagnostic decision making. We were surprised with the unreferenced
assertion that parents “do not have the clinical ability to evaluate the
temperature of their children and overvalue signs and symptoms, in order
to seek better care for children”. We would hope that parents are
effective advocates for their children. Given that children are often too
young to provide a history it is to the parents that physicians turn to
elicit symptoms. One potential limitation of our study is we did not
elicit parent’s views on the likelihood of serious bacterial infection and
compare them with the physicians’ views. In our view, the axiom, “parents
know best”, is frequently true and does need formal evaluation.
Competing interests:
None declared
Competing interests: No competing interests