Re: Clinicians’ gut feeling about serious infections in children: observational study
Van den Bruel, Thompson, Buntinx and Mant (2012) draw on a sample of young people presenting in primary care and discuss clinicians’ gut feelings about the severity of their infections. We note that there does not appear to have been any consideration given to potential clustering in the data; i.e. no indication is given of the number of clinicians who provided the clinical impressions or gut feelings. We expect that ratings made by one individual may have more in common with other ratings made by this individual compared with ratings made by other individuals.Unless every patient was rated by a different clinician, thus creating independent observations, this omission is likely to be of importance given that there were systematic differences between clinicians in terms of their likelihood of experiencing a gut feeling (more experienced clinicians were significantly less likely to experience a gut feeling).
Multilevel modelling approaches allow the hierarchical structure of data to be taken into account and would appear to represent an improvement over logistic regression analysis in the current context (see e.g., Goldstein, 2011; Snijders & Boskjer, 2012). As illustrated by LaHuis and Avis (2007), an advantage of multilevel modelling is that it allows consideration to be given to how attributes of raters influence their ratings while also taking into account characteristics of those rated. There is a substantial body of evidence highlighting the perils of not accounting for a hierarchical data structure when this is present (e.g Goldstein, 1997).
The following gives an indication of how a multilevel model might be implemented in the work of Van den Bruel et al. Patients can be considered to represent the first level of interest and clinicians the second. (The clinic in which the clinician works might be considered to represent a third level.) The outcome of interest is for each patient, whether or not the clinician had a gut feeling of serious illness. Also, for each patient, there is information about his/her presenting symptoms, his/her overall appearance, parental concern, and findings from a clinical examination; these represent possible explanatory variables at Level 1. For each clinician, information is available on his/her experience as a doctor; this represents a possible explanatory variable at level 2. A flexibility of multilevel modelling is that each clinician may have seen varying numbers of children.
To conclude, there is a substantial body of research evidence demonstrating the importance of taking into account the clusters which may exist in data. Multilevel modelling offers one approach to dealing with such clustering.
Goldstein, H. (1997). Methods in school effectiveness research. School Effectiveness and School Improvement, 8, 369-395.
Goldstein, H. (2011). Multilevel statistical models (4th ed). Chichester: Wiley. DOI: 10.1002/9780470973394.
Lahuis, D. M. & Avis, J. M. (2007). Using multilevel random coefficient modelling to investigate rater effects in performance ratings. Organizational Research Methods, 10, 97-107. DOI: 10.1177/1094428106289394.
Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modelling (2nd ed). London: Sage.
Van den Bruel, A., Thompson, M. Buntinx, F., Mant, D. (2012). Clinicians’ gut feeling about serious infections in children: observational study. British Medical Journal, 345:e6144. DOI: http://dx.doi.org/10.1136/bmj.e6144.
Competing interests: No competing interests