Intended for healthcare professionals

Endgames Statistical Question

Understanding the Hawthorne effect

BMJ 2015; 351 doi: https://doi.org/10.1136/bmj.h4672 (Published 04 September 2015) Cite this as: BMJ 2015;351:h4672
  1. Philip Sedgwick, reader in medical statistics and medical education1,
  2. Nan Greenwood, associate professor in health and social care service research2
  1. 1Institute for Medical and Biomedical Education, St George’s, University of London, London, UK
  2. 2Faculty of Health, Social Care and Education, Kingston University and St George’s, University of London
  1. Correspondence to: P Sedgwick p.sedgwick{at}sgul.ac.uk

Researchers investigated the effectiveness of patient controlled analgesia for patients presenting to emergency departments with pain from traumatic injuries. A randomised controlled trial with a parallel groups study design was used. The control treatment was usual care, with analgesia titrated by nurses according to hospital guidelines. The participants were patients aged 18-75 years presenting to the emergency department who required intravenous opioid analgesia for the treatment of moderate to severe pain resulting from traumatic injuries, and who were expected to be admitted to hospital for at least 12 hours. In total, 200 adults were recruited and randomised to the intervention (patient controlled analgesia; n=99) and control (routine care; n=101) treatment groups.1

The primary outcome was total pain experienced over the 12 hour study period, recorded using a visual analogue scale. Secondary outcomes included total amount of morphine used. The mean total pain experienced by the intervention group was lower than that experienced by the control group, although the difference was not significant. Participants in the intervention group used significantly more morphine than was administered in the usual care group. It was concluded that patient controlled analgesia provided a reduction in pain, albeit not significant, compared with routine care for emergency department patients with traumatic injuries. The researchers commented that the results of the trial may have been influenced by the Hawthorne effect.

Which of the following statements, if any, are true?

  • a) The Hawthorne effect is a change in behaviour as a response to observation and assessment

  • b) The nurses administrating routine care for the control group were prone to the Hawthorne effect

  • c) Participants in the intervention group were prone to the Hawthorne effect

  • d) Participants in the control group were prone to the Hawthorne effect

Answers

Statements a, b. c, and d are all true.

The aim of the trial was to determine whether patient controlled analgesia (intervention treatment) was more effective than routine care (control treatment) in patients presenting to emergency departments with moderate to severe pain from traumatic injuries. The researchers commented that the results of the trial may have been influenced by the Hawthorne effect. The Hawthorne effect is a non-specific treatment effect; it is a change in behaviour as a motivational response to the interest, care, or attention received through observation and assessment (a is true). The nurses administrating routine care to patients in the control group were prone to the Hawthorne effect (b is true), as were the participants in the intervention group (c is true) and those in the control group (d is true). As a consequence of behavioural changes by the nurses or participants the outcome measures, and therefore the results of the trial, may have been affected. However, in the above trial the extent of the Hawthorne effect and its effect on the results were not easily quantifiable.

The nurses caring for the patients in the routine care group would have been aware that they were being observed and that their patient management was being recorded—in particular the amount of morphine administered. If the nurses changed their routine behaviour as a consequence of the observation and measurement of their patient management in the trial, the Hawthorne effect would have occurred. Although the nurses were required to prescribe analgesia according to hospital guidelines, this may not have occurred in routine care for a variety of reasons. As is typical of most trials, the patient care received exceeds that given in routine clinical care.

The trial participants may have been prone to the Hawthorne effect for a variety of reasons. Patients allocated to the intervention group controlled how much analgesia they received. If they modified their behaviour—for example, by reducing the amount of analgesia used because they believed that their usage would be considered excessive and therefore unacceptable—the Hawthorne effect would have occurred. The participants in the routine care group were prescribed analgesia according to hospital guidelines. However, these participants may not have wanted the quantity of analgesia they were prescribed in the trial. If the participants in the control group changed their behaviour, for example, by accepting the analgesia as prescribed because they believed it was essential to maintain the allocated therapeutic regimen, the Hawthorne effect would have occurred.

The Hawthorne effect derived its name from a study of the psychological aspects plus physical and environmental influences in the workplace at the Hawthorne Plant of the Western Electric Company in Cicero, Illinois, during the 1920s. Workers increased their productivity when they were studied, but it declined when the study finished. The results implied that research participants may change their behaviour simply because of the attention they receive, regardless of the experimental manipulation. However, the original study claimed remarkable changes in productivity as a consequence of the Hawthorne effect. Subsequent re-analysis of the data showed that these claims were false and that, although the Hawthorne effect existed, changes in productivity were more subtle than originally suggested. Typically, the Hawthorne effect is described as a change in behaviour of the research participants in experimental or observational studies. However, the researchers or healthcare practitioners delivering patient care may also be prone to the Hawthorne effect, particularly if their performance or efficiency is being measured, as in the study above.

In the above trial the participants and the nurses delivering the routine care were not blinded to the treatment allocation. Knowledge of the treatment allocation may have contributed to the Hawthorne effect. However, the Hawthorne effect can exist even if a trial is double blind. Placebos or sham treatments are administered in trials to facilitate blinding. However, healthcare practitioners will typically know if they are delivering care to patients in a trial, and presumably their behaviour may change as a consequence. In a placebo controlled trial the Hawthorne effect may contribute to a series of non-specific treatment effects, collectively known as the placebo effect, resulting from the administration of a placebo.2 Other components of the placebo effect are the patients’ response to a therapeutic ritual (regular medical treatment) and their response to the patient-doctor interaction. Obviously, patients receiving the intervention will also experience these non-specific treatment effects. No doubt the association between the components of the placebo effect is complex. The placebo effect can be elicited by the provision of any therapeutic regimen and not solely in clinical trials that include a placebo treatment arm.2 Hence the Hawthorne effect may be present in any clinical encounter, regardless of whether the clinical care takes place as part of a trial.

The Hawthorne effect has considerable implications for the generalisability of results from research to clinical practice. Without doubt the behaviour of research participants is influenced by observation and measurement. However, it has been suggested that there is no single Hawthorne effect and that the terminology encompasses a variety of potential biases. To help understand the Hawthorne effect and elucidate the implications for clinical practice, greater exploration of how the experiences of patients and healthcare professionals differ between the research setting and routine practice is essential. Mixed methods, including qualitative and quantitative approaches, are possible ways to improve our understanding of the Hawthorne effect. Data triangulation, whereby data are collected using a variety of methods from different sources, may be a valuable way forward.

Notes

Cite this as: BMJ 2015;351:h4672

Footnotes

  • Competing interest: None declared.

References

View Abstract