Article Text
PDF
Abstract
Background Although the topic of identifying febrile children at risk of serious infections has been addressed by numerous research groups, identified predictors remain diverse and implementation of results in routine practice has been limited. The aim of this paper is to discuss the problems and challenges in advancing diagnostic research in febrile children.
Methods The characteristics and results of 35 studies identified from a systematic review on predictors for febrile children were evaluated.
Results Current diagnostic research is mainly performed in subpopulations, defined by age and temperature limits and in paediatric emergency settings, ignoring the role of primary care. It is characterised by a dichotomous approach of outcomes and a wide variability of potential predictors. Validation of results to other settings and impact studies of prediction rules on patient outcomes are scarce. In designing diagnostic studies on children suspected of serious infections focus is needed on all clinically relevant populations within the spectrum of primary care and emergency department settings. Consensus is also needed on the definition of fever, the concept of serious infection and the set of predictors to focus on. The heterogeneity of patients in different settings and countries stress the need for continuous updating of prediction rules in routine practice. Broad validation in different clinical settings and countries and impact analysis in routine care is essential.
Conclusions Scientists in the field of diagnosis of serious infection in children must agree on core design features to be incorporated in all studies in the area of diagnostic research in febrile children. This will improve evidence from future studies, and their generalisability and implementation in routine practice.
Statistics from Altmetric.com
Introduction
Serious infections are one of the most common presentations encountered among children attending emergency departments and primary care settings.1 ,2 The key task facing clinicians in these settings is distinguishing children who are more likely to have either serious infection (eg, meningitis, pneumonia), or complications of infection (eg, dehydration from viral gastroenteritis) from the vast majority with self-limiting or minor infections who can safely be managed at home. Regardless of marked falls in the incidence of serious bacterial infections as a result of vaccination3 parental anxiety about febrile illness in their children remains,4 and attendance rates at emergency or primary care are high. Missed and delayed diagnoses of serious infections still occur, adding to morbidity and mortality. Delayed diagnoses are a major reason for malpractice lawsuits in paediatric practice.5
There have been a number of landmark reviews and practice guidelines in this area,6,–,11 building on primary research going back to the development of the Yale and Rochester observation scores in the 1980s.12 ,13 As analytic approaches used in diagnostic research have evolved, clinical prediction rules for children with fever have also been developed.14,–,28 As members of a European research collaboration on improving the recognition of serious infections in febrile children, the authors recently published two systematic reviews which assessed the current state of the research on the predictors of serious infection in children in ambulatory care.29 ,30 The aim of this current study is to identify a number of deficits in the methodology and study design of the current primary literature in this area. We discuss the barriers to advancing diagnostic research in febrile children and use a methodological perspective to propose possible solutions to bridge the gap between research and clinical practice in this area.
Methods
For a full description of the literature search and the clinical contents please refer to previous publications.29 ,30 In short, we searched electronic databases (Medline, Embase, DARE, CINAHL), reference lists of relevant studies and contacted experts to identify articles assessing clinical features of serious infection in children. Studies were selected on the basis of six criteria: design (studies of diagnostic accuracy or prediction rules), participants (otherwise healthy children aged 1 month to 18 years), setting (emergency departments and primary care), outcome (serious infection), features assessed (assessable in ambulatory settings) and sufficient data reported.
Analysis
We characterised the studies based on patient inclusion, outcome, the number and type of clinical variables identified per study and their year of publication. Studies were categorised according to setting, with prevalence of serious infection as a proxy: less than 5% was defined as low prevalence, 5% to 20% as intermediate and more than 20% as high prevalence setting. We also categorised studies according to standards of evidence for developing and evaluating prediction rules as proposed by Reilly et al: derivation, validation (internally or externally), impact analysis (narrow and broad).31 Table 1 summarises the most important issues addressed, to guide the reader through this paper.
Identified problems of diagnostic research in febrile children and proposed solutions
Results and discussion
Patient selection
Online table 2 summarises the thresholds used for temperature and age as observed in the 35 primary studies on clinical predictors of serious infections in febrile children identified in the systematic reviews. The majority of studies included children between 3–24 months; only a quarter included patients up to adolescence.16 ,27 ,32,–,37 Children with a temperature above 39.5°C were included by all but one of the studies.36 In the range 38.0–39.5°C less agreement of inclusion was observed.
Selection of more homogeneous groups of patients by for example age or temperature facilitates diagnostic research and simplifies conclusions. These selection criteria, however, limit the external validity of study findings to children outwith these thresholds. The risk of serious bacterial infections is now similar in young and older children (as a result of successful vaccination strategies).3 We propose that primary studies should now include age as a predictor itself, rather than selecting the population within certain age limits, but still defined as children (age <16 years).
Parents worry about fever, and frequently use antipyretics to reduce fever prior to consultations.4 Using certain temperature thresholds at initial consultation as inclusion criteria makes a prior assumption about the relationship between fever and infection severity or that the antipyretic response in children is less marked in those with serious infections compared to those with minor infections. We do not think that such an assumption holds true.38 We think that inclusion based on parental report of raised temperature39 or a measured temperature above normal values8 would allow generalisability of results to the broader clinical diagnostic dilemma of febrile children. Furthermore, such a pragmatic definition of fever overcomes the difficulties of measurement of fever, inaccuracies of devices and differences between different methods or sites such as tympanic, rectal or oral assessment of temperature.8 Although there is some evidence that the height of fever in some settings increases the risk of serious infection29 we need to incorporate issues of general appearance and setting to define children at risk for serious infection in addition to temperature alone.
Selection and definition of predictors
Heterogeneity of selected predictors
We summarise the 26 predictors found in the systematic reviews and the diagnostic studies on serious infections in febrile children in which they were identified in online table 3. This shows a wide variability in the number and type of identified predictors, as well the paucity of predictors shared between studies. First, selection criteria for patients and different outcome definitions may cause heterogeneity in patients and distribution of types of serious infections between studies.40 Second, variability may occur because identical definitions for predictors are not used, and not all potential variables are studied. The variability in selection and definition of predictors limits the ability to combine information as a pooled diagnostic accuracy and detracts study results from their true validity. The meaning of symptoms varies widely within and between cultures, and many physical signs have very low interobserver agreement.41 Using more objective candidate clinical predictors such as vital signs (eg, temperature, heart rate, respiratory rate), or those that have a broadly agreed definition may reduce the heterogeneity. Further, the heterogeneity underlines the need for broad validation studies of prediction rules in external populations.31
Confirming clinical value
Selection of predictors is guided by the analytic strategies employed. Given the large number of potential predictors and the relative low prevalence of serious infections, data-driven analytic approaches may result in a number of predictors identified by chance,42 ,43 but miss true predictors. This may have caused limited external validity of previous studies.15 ,42 It has been suggested that the design of future studies should focus on the added diagnostic value of new variables (eg, inflammatory markers such as procalcitonin or interleukins) to previously known predictors, rather than attempting to build a completely new rule.44 We therefore challenge clinicians to collaborate with researchers to define the set of important and practical predictors which should be included. The ‘red flag’ signs identified in the review29 are present in only a small proportion of all children presenting with fever, and thus are highly specific but poorly sensitive.45 In addition to searching for the perfect alarm symptom (red flag), it may be useful to identify a set of frequent occurring symptoms that make poor outcomes less likely if all are present (eg, ‘green flags’).
Diagnostic value of ‘new’ predictors and ‘time course’
With the widespread use of telephone triage and advice systems there is a need for studies to explore the reasons why parents seek medical attention. The systematic reviews identified just one study including parental concern as a significant predictor of serious infection.27 Moreover, symptoms elicited from a parent often have different textual meaning to the same word used in a clinical context.46 ,47 The potential diagnostic value of the parental perspective has been overlooked in many research studies and should be included in the design of future studies.
It is inevitable that some children with serious infection will be missed at initial diagnosis. Providing parents with the information and empowerment to return to seek medical care if their child deteriorates is therefore an important diagnostic strategy, yet one that has rarely been studied. None of the studies identified in our systematic reviews examined the value of safety-netting. Furthermore, the debate on the definition and contents of ‘safety-netting’ is ongoing.48
Outcomes
Widely used prediction rules for conditions such as ankle injuries49 ,50 aim to predict fairly uniform outcomes (eg, presence of ankle fracture). In contrast, the diagnostic outcomes in studies of children presenting with fever are far more heterogeneous. Some of the studies identified in our systematic reviews used a composite outcome of serious bacterial infections, defined by positive bacteriological cultures from normally sterile sites (n=20).12 ,14 ,15 ,18,–,20 ,23 ,26,–,28 ,32 ,35 ,36 ,51,–,57 Others focused on specific outcomes such as pneumonia (n=2),58 ,59 meningitis (n=3),16 ,24 ,25 septicaemia (n=2),33 ,37 gastroenteritis (n=2)21 ,60 or bacteraemia (n=6).22 ,34 ,61,–,64 Based on the selection of outcomes and their definition we identified four problems in translating findings to clinical practice: difficulties in establishing a reference standard, diagnostic shift in what is considered serious, failure to recognise the different diagnostic needs in different clinical settings and use of dichotomous outcomes.
Lack of reference test to confirm bacterial origin
Diagnoses confirmed by a reference standard optimally avoid verification bias,65 but this is rarely practical in studies carried out in primary care and emergency department settings. Also, invasive testing for children who have a low probability of serious infection does not seem ethical.66 ,67 Absence of a reference standard, however, is often considered to affect the diagnostic quality of research in this area. Most diagnostic studies in children are performed in acute clinical settings and, therefore, have to be pragmatic in outcome assessment. A composite reference combining multiple test results, following children over a period of time, or normalisation of temperature or other symptoms68,–,70 may be alternatives for valid reference standards.
The changing definition of serious infection
In addition to the lack of reference test to confirm bacterial infections as discussed above, invasive bacterial infections such as meningitis nowadays have a low incidence in most western paediatric settings. Therefore, it is difficult to derive and validate prediction rules. In a recent Australian study of 15 781 children presenting to a paediatric emergency department, there were too few cases of meningitis to be included in the prediction rule for serious infection, limiting its usefulness.69 Given the rarity of invasive bacterial infection, the definition of ‘serious’ has drifted recently to include a less severe spectrum of infections. Agreement is needed on what is ‘serious’—defined by invasiveness (ie, positive cultures), the need for hospitalisation or intervention (eg, for parenteral treatment or oxygen), or the risk of immediate or longer-term complications (eg, risk of renal scarring in urinary tract infection)—and how well current definitions reflect this.
The overall aims of diagnosis may depend on setting and prior risk of diagnosis. In low prevalence primary care settings clinicians are more interested in ruling out serious infection. Identifying children who need referral to secondary care for further assessment or therapy is more important than confirming a particular diagnosis. In contrast, in higher prevalence settings such as the emergency department, clinicians evaluate the child for a (final) diagnosis, have access to diagnostic testing and place greater emphasis on ‘ruling in’ diagnostic outcomes. Future studies in this field need to make these different outcomes explicit.
Heterogeneous outcomes
Most research studies used dichotomous outcomes, that is, the presence or absence of serious infection. This may cause two sets of problems. First, the positive outcome is usually a composite of all serious infections together (as noted above), assuming that identified predictors predict the different serious infections in a similar way. This is counterintuitive; respiratory signs, for example, predict pneumonia but would not be related to urinary tract infections. Conditions with the highest frequency usually dominate within such composite outcomes,69 so even studies using a composite outcome for bacterial infections result in predictions for pneumonia being the most frequent bacterial infection nowadays.
The dichotomous approach also overlooks children with an intermediate level of diagnosis in terms of severity or probability. Clinicians are more interested in decision support for patients in this ‘grey’ area, rather than those who obviously do or do not suffer from serious infections. Most diagnostic research studies either fail to report the results of children in these intermediate categories, or aggregate them with the serious group.71 Polytomous logistic regression modelling may partly solve this problem as it simultaneously models predicted probabilities of multiple diagnostic outcome categories.72 ,73 However, there is a need to discuss how to interpret or weight these multiple outcome predictions for clinical practice. Should we base management on a low threshold risk for the most serious but most rare outcome (eg, meningitis/septicaemia), or on the highest threshold risk for a less serious but more frequently occurring disease?
Generalisability to different clinical settings
Setting with differences in prior risk
The majority of children with serious infections who seek help from healthcare professionals attend primary care. An estimated 5% of children are referred from primary care to secondary care,74 and of those seen in emergency departments 15% to 30% are admitted to hospital.75 ,76 The systematic review, however, identified only a minority (6/35) studies conducted in low prevalence settings, with only a single study performed in primary care.27 This highlights the major mismatch between the numbers of studies performed and the settings children are most often evaluated in. We urgently need studies covering all clinically relevant populations: children at risk for serious infections attending casualty or children attending primary care. In addition, continuous validation and updating of management approaches remains necessary to monitor changes in disease (such as during pandemic flu) or changes in parental behaviour (in response to publicity).40
Mismatch of the diagnostic research approach to the clinical diagnostic process
We observed a mismatch between the analytic approach used to generate clinical decision rules, and the actual diagnostic processes in clinical practice. Clinical diagnosis is a dynamic rather than static process; clinical features evolve as a disease progresses, and time is an important diagnostic tool.77 Although stepwise analysis of predictors from history, examination and additional tests may mirror the phased investigation in clinical practice,65 it is difficult to replicate the time factor by conducting sequential analyses of cross-sectional data. None of the studies identified in the systematic review included any repeated assessment of children or examined the impact of speed of illness onset. As no prediction rule will achieve a 100% sensitivity or specificity, an explicit and comprehensive reassessment plan, that is safety-netting, is important when implementing a decision rule.48 However, the optimal content and way of delivering safety-netting advice is unclear.
Lack of impact on patient outcomes or implementation
Reilly reviewed standards of evidence for developing and evaluating prediction rules and identified five essential steps, as presented in online table 4.31 Matching the 35 studies to these 5 steps illustrates the paucity of validation or impact studies. The results of only six studies have (partly) been validated by other researchers in new settings; three studies included in the systematic reviews19 ,54 ,56 externally validated results of a previous study.12 ,51 ,52 One study assessed the impact of testing for procalcitonin on process outcomes.78 We are aware of one impact analysis on patient outcome of a prediction rule for febrile children, with disappointing results.17 These deficiencies in implementation or impact analyses strongly illustrate the disconnection between diagnostic research and clinical practice. They risk premature or even inappropriate adoption of prediction rules, or of new diagnostic technologies. Bridging this gap is essential but requires closer collaboration between researchers and practitioners to identify the knowledge gaps in the entire diagnostic pathway.
Conclusions
Although the topic of children at risk for serious infection has been addressed by numerous research groups, results remain diverse and implementation of research findings in routine practice is limited. In a secondary analysis of the most recent state of the art systematic reviews in this field, we do not attempt to reproduce the existing well cited literature on study design and reporting (Standards for reporting of diagnostic accuracy (STARD), quality assessment of studies of diagnostic accuracy (QUADAS)).79 Rather, our analysis highlights a total of 10 study design features specific to primary research on children with serious infections. These features spanned patient selection, selection and definition of predictors, outcomes and generalisability. To improve diagnostic research in febrile children, we propose several solutions that would improve the design. We challenge clinicians and diagnostic researchers to focus diagnostic research in febrile children on all clinically relevant populations within the spectrum of primary care and emergency department settings. We have proposed consensus definitions of some basic components of paediatric care, such as the definition of fever, the concept of serious infections and the set of predictors to focus on. These will in turn facilitate studies evaluating the added value of new diagnostic technologies. The heterogeneity of patients in different settings and countries and changing epidemiology of outcomes stress the need for continuous updating of prediction rules in routine practice. In-depth research of diagnostic safety-netting might generate new insights in and contribute to diagnostic management of the child with serious infections. Finally, to achieve implementation of study results in routine care, we need broad validation of results in different types of clinical settings, and impact analysis in routine care.
What is needed now? We call on researchers, journal editors and clinicians who conduct, report and interpret studies on diagnosis of serious infection in children to further discuss and to agree a consensus of opinion to which should be considered as core design features to be incorporated in all studies in the area of diagnostic research in febrile children. This will improve evidence from future studies, and their generalisability and implementation in routine practice.
Acknowledgments
The authors wrote the paper on behalf of the European Research Network on recognising serious Infection (ERNIE). The principal ERNIE investigators are: Marjolein Berger, Frank Buntinx, Monica Lakhanpaul, David Mant, Henriette Moll, Rianne Oostenbrink, Richard Stevens, Matthew Thompson, Ann Van den Bruel and Jan Verbakel.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Web Only Data - This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
-
Funding RO is financed by an unrestricted grant from Europe Container Terminals B V and by a grant from the European Society of Pediatric Infectious Diseases. MT is supported by NIHR HTA Project 07/37/05: ‘Systematic review and validation of clinical prediction rules for identifying children with serious infections in emergency departments and urgent-access primary care’.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.