Rates of medication errors among depressed and burnt out residents: prospective cohort study
BMJ 2008; 336 doi: https://doi.org/10.1136/bmj.39469.763218.BE (Published 28 February 2008) Cite this as: BMJ 2008;336:488- Amy M Fahrenkopf, instructor of paediatrics1,
- Theodore C Sectish, associate professor of paediatrics2,
- Laura K Barger, research fellow3,
- Paul J Sharek, assistant professor of paediatrics2,
- Daniel Lewin, assistant professor of psychiatry and paediatrics4,
- Vincent W Chiang, assistant professor of paediatrics1,
- Sarah Edwards, project coordinator3,
- Bernhard L Wiedermann, associate professor of paediatrics4,
- Christopher P Landrigan, assistant professor of paediatrics and medicine13
- 1Harvard Medical School, Department of Medicine, Children’s Hospital Boston, Boston, MA 02115, USA
- 2Stanford Medical School, Lucile Packard Children’s Hospital, Palo Alto, CA, USA
- 3Division of Sleep Medicine, Brigham and Women’s Hospital, Boston, MA, USA
- 4George Washington University Medical School, Children’s National Medical Center, Washington, DC, USA
- Correspondence to: A M Fahrenkopf amy.fahrenkopf{at}tch.harvard.edu
- Accepted 8 December 2007
Abstract
Objective To determine the prevalence of depression and burnout among residents in paediatrics and to establish if a relation exists between these disorders and medication errors.
Design Prospective cohort study.
Setting Three urban freestanding children’s hospitals in the United States.
Participants 123 residents in three paediatric residency programmes.
Main outcome measures Prevalence of depression using the Harvard national depression screening day scale, burnout using the Maslach burnout inventory, and rate of medication errors per resident month.
Results 24 (20%) of the participating residents met the criteria for depression and 92 (74%) met the criteria for burnout. Active surveillance yielded 45 errors made by participants. Depressed residents made 6.2 times as many medication errors per resident month as residents who were not depressed: 1.55 (95% confidence interval 0.57 to 4.22) compared with 0.25 (0.14 to 0.46, P<0.001). Burnt out residents and non-burnt out residents made similar rates of errors per resident month: 0.45 (0.20 to 0.98) compared with 0.53 (0.21 to 1.33, P=0.2).
Conclusions Depression and burnout are major problems among residents in paediatrics. Depressed residents made significantly more medical errors than their non-depressed peers; however, burnout did not seem to correlate with an increased rate of medical errors.
Introduction
Depression and burnout are highly prevalent among residents worldwide and across specialties.1 2 3 4 5 6 7 8 9 Studies have found rates of burnout in residents to be between 41% and 76%,9 whereas rates of depression range from 7% to 56%.1 2 10 11 The stresses of resident training, including sleep deprivation and lack of leisure time, are the most commonly cited explanations.1 2 3 4 5 6 7 8 Despite the frequency of burnout and depression few studies have quantified their relation with patient safety. According to the Institute of Medicine, between 44 000 and 98 000 patients die each year in the United States as a result of medical errors,12 and over 400 000 preventable adverse drug events may occur.13 Adverse events are also common in the United Kingdom, occurring in more than 10% of hospital admissions; as many as half of these adverse events might have been preventable.14 Recent studies have shown that the working conditions of healthcare providers—including sleep deprivation15 16 17 and overwork18 19—contribute substantively to this problem.20 The mental health of healthcare providers has been less well studied. A few studies have examined the relation between burnout in residents and self reported medical errors,1 11 21 but the reported relation has not been validated; it is unclear whether burnout is associated with more medical errors or whether burnt out residents simply perceive themselves to be making more errors. Similarly, the relation between depression and medical errors has not been quantified systematically.
We studied residents specialising in paediatrics in three centres to determine the prevalence of depression and burnout and whether a relation exists between these disorders and medication errors. We also collected depressed and burnt out residents’ self reports of their health and medical errors.
Methods
Data for this study were collected as part of the paediatric duty hours study, which aimed to measure the effects of the Accreditation Council for Graduate Medical Education duty hour standards on the work hours, safety, health, and educational experiences of residents. The study was advertised to residents in both paediatrics and medicine-paediatrics at three institutions: the Children’s Hospital Boston in Boston, Massachusetts; the Lucile Packard Children’s Hospital in Palo Alto, California; and the Children’s National Medical Center in Washington, DC. The study was advertised by fliers, emails, and announcements made during morning rounds. All residents in paediatrics and medicine-paediatrics were eligible to participate, except those on extended leave or those working exclusively in non-paediatric settings at the time of the study. Residents who agreed to participate in the study gave written informed consent. Participants were aware that we were collecting data on their health, safety, and performance during the study. Precautions were taken to secure confidentiality, including the assignment of coded identification numbers and secure storage of data. Participants were assured that only investigators would have access to the code key and that they would face no disciplinary action for any errors detected, nor would errors be analysed on an individual basis. They were also informed that the only instance in which confidentiality would be broken would be if they were an immediate danger to themselves or others—that is, showed suicidal or homicidal ideation. Investigators who were also officials of the residency programme were not given access to the code key.
Data collection
Participants logged their daily work and hours of sleep from mid-May through to the end of June 2003. They also completed a validated questionnaire on their health, quality of life, and self reported medical errors, which had been extensively piloted and used in multiple studies.22 23 24 We used two standardised screening tools—the Harvard national depression screening day scale25 and the Maslach burnout inventory26—to assess the prevalence of depression symptoms and burnout. The Harvard national depression screening day scale is a 10 question, validated screening tool for depression, which we chose for its brevity, ease of administration, and sensitivity for major depression. A score in the upper third (≥9) has been found to be 94% specific and 95% sensitive for a major depressive episode, similar to both the Beck depression inventory and the Zung self rating depression scale, which are twice the length.25 The Maslach burnout inventory is a 22 question, validated questionnaire that is considered the ideal instrument for measuring burnout26; in this study we used a version specific to medical staff. Burnout is a syndrome of mental exhaustion and personal detachment that develops in response to chronic occupational stress. Burnout differs from depression in that the symptoms affect the individual’s relation to work and usually spare their personal life. The Maslach burnout inventory measures three components of burnout: emotional exhaustion, depersonalisation, and personal achievement. By convention, scores for personal achievement are not included in the total score as this domain is considered an independent subscale.26 We defined participants as being depressed if they scored more than 9 on the Harvard national depression screening day scale and as being burnt out if they had a “high” combined score for emotional exhaustion (≥27) and depersonalisation (≥10) on the Maslach burnout inventory, in accordance with guidelines of the respective scales. For the purposes of this study we considered participants not to be depressed who reported a history of depression or who were actively being treated for depression but did not score more than 9 on the Harvard national depression screening day scale.
Concurrently, at two of the three study sites (Children’s Hospital Boston and Lucile Packard Children’s Hospital) we used well established, comprehensive methods to prospectively collect data on medication errors on wards.27 This included a daily review of the charts and medication orders for all patients on the surveyed wards and a review of solicited and voluntary error reports by staff. Errors collected could be linked to participants and could subsequently be classified as having involved residents who were depressed or burnt out. We were unable to collect prospective data on medication errors that could be linked to participants at the Children’s National Medical Center and therefore we excluded the centre from this facet of the study. Data on errors were collected solely on wards; we did not include intensive care units or ambulatory settings.
A team of nurses and physicians were hired and intensively trained in the collection of data on medication errors using standardised methods. They collected daily reports of all medication errors that occurred on studied wards from clinical staff and reviewed all charts and medication orders using structured data forms. Medication orders were considered erroneous whether or not they were intercepted in the medication administration process. The data extractors were unaware of the specific aims of the study or the status of the participants for depression or burnout. For each suspected medication error the data extractors recorded a description of the event and whether the error involved a participant. Blinded reviewers (AMF and CPL) then independently categorised each incident as an adverse event, potential adverse event, or error with little potential for harm. Adverse events were further deemed preventable or non-preventable. Disagreements were resolved by discussion; before these discussions we calculated the inter-rater reliability with the κ statistic. Box 1 lists the definitions used to classify incidents.
Box 1: Definitions of terms used in the study
Medication error—any error in the ordering, transcription, or administration of a medication, whether harmful or trivial
Errors with little potential for harm—a medication error that has little potential for harm because it is a minor error or so incomplete as to make transcription or administration impossible
Potential adverse drug events—a medication error that has major potential to cause harm but does not because it is intercepted before reaching the patient or because the patient has sufficient physiological reserves to absorb the error without apparent injury
Adverse drug event—any injury due to medical management
Non-preventable adverse event—unavoidable injury resulting from appropriate medical care
Preventable adverse event—injury due to a non-intercepted serious error in medical management
Statistical analysis
We used Fisher’s exact tests to compare categorical responses of depressed and burnt out residents with those of their counterparts. Multivariate logistic regression analyses were then done controlling for sex, race, age, marital status, postgraduate year, and site of residency. Statistical tests were two tailed; we considered P values less than or equal to 0.05 to be significant.
Using a cluster adjusted Poisson analysis we compared the number of errors attributable to the different resident subpopulations (depressed residents v non-depressed residents and burnt out residents v non-burnt out residents) per resident month (four weeks). We determined confidence intervals and P values using the cluster analysis. Inter-rater reliability for the classification of events was calculated before discussion using the κ statistic. We used SPSS version 14.0 for statistical analyses.
Results
Overall, 123 of 246 eligible residents (50%) participated. No statistical difference was found between age, sex, or postgraduate year for participants and non-participants. Table 1⇓ lists the personal data for participants.
Overall, 125 errors were identified on review of 10 277 orders during the study (total error rate 1.2%). Participants wrote a total of 6078 orders, with 45 errors (error rate 0.7%). Among these errors 0 were for adverse drug events, 28 for potential adverse drug events, and 17 for errors with little potential for harm. The κ was 0.88 for inter-rater reliability of event classification. Box 2 gives representative examples of medication errors captured.
Box 2: Examples of medication errors
Errors with little potential for harm
Order written for tylenol for patient with osteomyelitis but no frequency provided
Order written for cefepime for patient with cystic fibrosis but no route provided
Potential adverse drug events
Order written for intravenous potassium chloride bolus for patient with cancer but no concentration or rate provided
Order written for morphine on the wrong patient’s order form
Preventable adverse drug events
Example would be (none was detected) medication given to patient with known allergy, who then experienced anaphylaxis
Depression
On the basis of scores on the Harvard national depression screening day scale 20% (n=24) of residents were found to be at high risk for depression. Of those screening positive, 26% (n=7) had a history of depression. Eleven participants were taking antidepressants at the time of the study, three of whom screened positive for depression. No association was found between depression and age, sex, year of residency, ethnicity, marital status, or logged sleep and work hours.
On active surveillance at two of the sites the medication errors per resident month for depressed residents was 1.55 (95% confidence interval 0.57 to 4.22) and for non-depressed residents was 0.25 (0.14 to 0.46, P<0.001; fig 1⇓). The cohort of 19 depressed residents made a total of 24 errors (0 preventable adverse drug events, 12 potential adverse drug events, and 12 errors with little potential for harm). By comparison the cohort of 82 non-depressed residents made a total of 21 errors (0 preventable adverse drug events, 16 potential adverse drug events, and 5 errors with little potential for harm). Table 2⇓ shows the total error count per resident. Figure 2⇓ shows the residents’ self reports of medical errors.
Depressed residents were significantly more likely to report their health as poor (17% (n=4) v 4% (n=4), P=0.05), to report working in an impaired condition more than twice in the previous month (44% (n=10) v 20% (n=19), P=0.03), and to report difficulty concentrating at work (70% (n=16) v 38% (n=37), P<0.01; fig 3⇓). Overall, 96% (n=23) of residents who met the criteria for depression also met the criteria for burnout.
Burnout
Ninety two residents (75%) met the criteria for burnout. Of those who were burnt out, 23 (25%) also met the criteria for depression. No association was found between burnout and age, sex, year of residency, ethnicity, marital status, or logged sleep and work hours.
Rates of errors per resident month detected on active surveillance did not differ significantly between burnt out and non-burnt out residents: 0.45 (0.20 to 0.98) compared with 0.53 (0.21 to 1.33, P=0.4; fig 1⇑). Figure 2⇑ shows residents’ self reports of medical errors. Burnt out residents were significantly more likely to report having made a “significant” error over the previous three months as a result of sleep deprivation: 29% (n=26) v 10% (n=3), P=0.05. Burnt out residents also reported a higher mean number of errors over the previous month (2.3 v 1.0, P=0.02).
Burnt out residents were significantly more likely to report difficulty concentrating on work (57% (n=51) v 7% (n=2), P<0.001) and concern about being depressed (37% (n=34) v 16% (n=5), P=0.04; fig 4⇓).
Discussion
In this tricentre study 20% of residents were depressed and 75% were burnt out. On active surveillance those who were depressed made more than six times as many errors in medication as their non-depressed peers. Furthermore, those who were depressed or burnt out reported poorer health and higher error rates than those who were not burnt out or depressed. This prospective study of burnout and depression among residents at three centres correlated the mental health of house staff with objective data on rates of medication error. Our finding that depressed residents had a medication error rate six times higher than that of non-depressed residents indicates that mental health may be a more important contributor to patient safety than previously suspected.
The prevalence of depression in our cohort of residents was nearly twice that expected in the general population28 and was near the mid-point of previously reported rates of depression among residents (range 7%-56%).1 2 10 11 Comparisons with rates of depression in the general population should be made with caution; however, given the infrequency with which residents reported a history of depression, it seems that residency itself may be associated with the onset of depression in a sizeable number of residents. Other authors have suggested that the commission of errors may itself lead to depression, making the doctor a “second victim” in an apparent vicious cycle.18 29 Nearly half of the depressed residents seemed unaware of their depression, despite being doctors, and only a small number were receiving treatment. These findings are sobering and warrant not only further investigation but also regular screening and treatment efforts in training programmes.
Burnout, although more pervasive than depression, had no objectively measurable association with medication errors. This is an important finding, as burnt out residents reported making significantly more medical errors than their non-burnt out colleagues in several studies, including our own. It is difficult to determine whether non-burnt out residents underestimate their error rates or whether burnt out residents overestimate their error rates or make an increased number of error types that were not captured in this study (for example, diagnostic errors). As in other studies, nearly all the depressed residents were burnt out, begging the question which came first?1 9 11 The high burnout rate in this study, which is consistent with that in other studies, also raises questions about whether current methods of doctors’ training generate avoidable stress that is detrimental to the health of residents.
Our study has several limitations. To do our statistical analysis assuming a Poisson distribution we had to make two key assumptions: that the resident workload was evenly distributed across the resident subpopulations and that the depressed or burnt out residents were evenly distributed across the wards for which orders were reviewed. We have data that support the first assumption from the resident work logs, showing no statistically significant difference in sleep or work hours between the different populations of residents. For the second assumption it is statistically highly unlikely that such a chance distribution could explain the differences observed, given the magnitude of difference observed between depressed and non-depressed groups; the likelihood of this cannot be entirely excluded, however. We did consider the possibility that one or two outliers could be responsible for the higher error rate among depressed residents. On review of the error data, errors were fairly evenly distributed across resident groups, with the exception of one resident responsible for 11 errors, who was both burnt out and depressed. We undertook a cluster adjusted analysis to tackle this problem. In addition, we reanalysed our data in a sensitivity analysis to see if our primary results changed if this resident was excluded and found that although the number of errors per resident month was reduced for both depressed and burnt out residents, the statistically significant differences between depressed and non-depressed residents persisted. The difference between burnt out and non-burnt out residents also remained unchanged, with no statistically significant difference.
Secondly, this study focused solely on residents in paediatrics, so the manner in which depression and burnout may affect house staff in other specialties is unclear. Although we have no reason to postulate that the relation between residents’ depression and error rates would be unique to paediatrics, further studies are needed to quantify the relation across specialties. Thirdly, we were unable to use data on errors from one of our three sites (Children’s National Medical Center), as the data were collected without the coded identifiers that allowed us to link errors to specific residents. Although we had sufficient power to show a significant link between depressed residents and medication errors on the basis of data from two centres, the additional data would have contributed to the robustness of the study. Fourthly, we collected our data before the implementation of any work hour limits for residents in the United States. Despite the fact that the link between depression and medical errors is unlikely to be affected by the change, the prevalence of mental health disorders could possibly have been affected by this intervention. Effects of reduced work hours on residents’ quality of life is unclear30; however, recent studies suggest that changes to work hours in the United States significantly decreased burnout scores but did not alter rates for depression.10 11 Lastly, whereas the number of residents who volunteered to participate in this study was high, particularly considering the time commitment required, and although available personal data suggest that our population of residents was typical of American paediatric residents,31 our population may have differed from the national cohort of residents in certain non-measurable respects. Participants signed informed consent and were aware that we were collecting data on work hours, mental health, and medication errors, but they were not aware of the specific hypothesis of this study. Thus we have no reason to believe that they would have chosen to participate or not on the basis of whether they were depressed or burnt out, and data on errors were collected by investigators blinded to the residents’ depression and burnout status.
This study raises important ethical concerns. Twenty four participants were found to be at high risk of major depression yet they could not be approached and encouraged to seek treatment owing to the confidentiality agreement, as none of them expressed active suicidal or homicidal ideation. Scoring positive for depression in the absence of expressing suicidal or homicidal ideation was not deemed adequate to break the confidentiality agreement. The general results of the study were shared with the three directors of the residency programmes in an effort to improve screening and treatment options, but it is unknown whether any of those residents received appropriate medical care. After the study had finished depressed residents were found to make significantly more medical errors yet this was unknown at the time of data collection and hence they could not be approached about their errors, nor could their attending doctors be notified of the need for more careful supervision. Without assured confidentiality it is unlikely that participants would be as candid in their responses in a research setting; however the presence of significant numbers of depressed residents on the paediatric wards, committing medical errors at a high rate, is cause for concern.
Our results highlight the need for better research on the mental health of doctors. Efforts should be made to screen residents for signs and symptoms of depression and to ensure proper treatment. Additional research is necessary to define better how depression and burnout affect residents and patient care in other specialties and to define better the causal relation between depression and errors. More information is also needed on the mental health of senior doctors. Lastly, rigorous trials are needed on interventions to improve the mental health of trainees.
Mental health problems are associated with decreased quality of life and loss of productivity in the workplace. Depressed healthcare providers may also put patients at risk of unintentional harm. Our study adds to an increasing body of literature showing the substantial relation between healthcare providers’ working conditions and health and the safety of patients and residents.15 16 17 18 19 20 22 23 32 33 Further efforts to study and improve the working conditions and mental health of doctors should be a priority.
What is already known on this topic
Depression and burnout are highly prevalent in doctors in training
Burnout is associated with a higher rate of self reported errors among residents
What this study adds
Depressed residents in paediatrics were more than six times as likely to make errors in medication as their non-depressed colleagues
Burnout did not seem to be associated with higher rates of medication errors
Footnotes
We thank the residents for their time and the nurses and doctors who helped us in our efforts to collect the data.
Ethical approval: This study was approved by the institutional review boards at the three participating hospitals.
Contributors: AMF and CPL conceptualised and designed the study, rated all medication errors, analysed and interpreted the data, drafted the paper, and approved the final version. PJS, DL, and CPL supervised collection of data on participants at the three sites. SE assisted in data analysis. All authors participated in designing the study, revising the manuscript critically for important intellectual content, and approving the final versio. AMF is the guarantor.
Funding: This study was supported by grants from the Agency for Healthcare Research and Quality (K08 HS13333) and from the Departments of Pediatrics at the Lucile Packard Children’s Hospital, the Children’s National Medical Center, and the Children’s Hospital Boston.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.