Improving compliance with requirements on junior doctors' hours
BMJ 2003; 327 doi: https://doi.org/10.1136/bmj.327.7409.270 (Published 31 July 2003) Cite this as: BMJ 2003;327:270
Data supplement
Figure A - as supplied by author
- Posted as supplied by author
Evaluating workload using control charts
Posted as supplied by author
Evaluating workload using control charts
An audit of workload at night provided an opportunity to create statistical process control (SPC) charts as a means of assessing the performance of the night rota and the effects of change This paper is a supplement to the paper "Same cards, New Deal; a bid for compliance" . It describes the methods of data collection and analysis, and the construction of so called X- and R-charts as an example of control charts for the analysis of systems in health care. Conventional summary statistics for the assessment of change have been published in the companion paper.
What are control charts?
When studying events that occur in a natural sequence over time, as many clinical events do, it is useful to preserve the dimension of time in the presentation and analysis. Mortality statistics as survival curves are a good example. Statistical process control (SPC) charts that analyse variation over time were introduced as a tool for understanding, managing and improving manufacturing systems by Shewhart, a statistician working in the 1920s. His pupil Edwards Deming and others have applied the approach to many spheres of activity (1,2,3) and SPC has begun to make an appearance in the evaluation of health systems (4).
In essence, a control chart plots a chosen measure of performance against time with an average and control limits based upon 3 standard deviations (SDs). The choice of 3 SDs is simply based on operational experience. 2SDs throws up too many "outliers" (1 in 20) whereas 3SDs (1 in 371) provides economically practical limits to guide action. The limits are there to help discriminate variation inherent in the system design (in the language of SPC, "common cause" variation) from signals that indicate outliers deserving of special investigation to establish cause ("special cause" variation). Improvement is about reducing variation, "special cause" by addressing the outliers and gaining greater stability, common cause by redesign of the system. The choice of chart and the method of calculation of limits depend on how the data has been collected and whether the data are measurements or attributes (1,2,3).
The present study used a pair of charts called X- and R-charts for single measurements. These use the moving range (MR), gaps between successive observations, as the measure of variation and method of calculating SD (rather than the usual square root of the variance from a mean value). A series of constants, dependent on the number of values used to calculate the MR, are available (1,2,3) for converting MRs into equivalent SDs. The X-chart plots individual values against time with a mean and upper and lower control limits (UCL, LCL = mean +/- 3SDs). The sample size for the MR is n=2 and SD = mean MR/1.128. The R-chart plots the MR to gauge serial trends in variability, the critical measure for evaluation of performance. For a sample size n=2 the SD of the MR is always 1.089. X- and R-charts should be viewed and interpreted together, the R-chart first to assess variation and the X-chart to evaluate average performance.
Methods
Data Collection
The process of data collection for the study was designed to be consistent with existing operational practices, readily understood by staff and to make minimal extra demands. The senior nurses on call at night (Clinical Site Practitioner) (CSP) already kept their own timed narrative record of ward events. After perusal of a sample of these records, it was decided to use these for the study, supplemented by independent data collection by two medical students who were employed to "walk the wards" from 10pm to 8am. They were asked to list every ward event using a structured data-collection sheet that reflected the staffing structure of the night-rota. They documented the grade of staff attending the wards (SHO, SpR or staff-grade, CSP), the time of arrival and departure and a short description of the reason for attendance. Ward nurses were asked to provide this information. The medical students added to it by direct observation and communication with medical staff. The whole process of data collection and recording was piloted for a week to provide practice for staff and to refine the process. Data was then collected for 28 consecutive nights before (Phase 1) and again after changes to the rota (Phase 2).
Assembly of nightly records
A tally chart, resembling a Gantt chart (see Figure 1 of published paper), was devised to allow assembly of data on each night’s work-load. Staff were listed down the left hand side. Time was represented across the top from 10pm to 8am divided into ½ hour intervals. Onto this grid at the end of the night, students and CSP, working together, reviewed the night’s records and identified and graded each event (1= should be left to the day to 4 = emergency, do now see Box 2 published paper). They then drew a time-line to represent each event and link it to the relevant member of the night rota, 12 in Phase 1 (see Figure 1 of companion paper) and 5 in Phase 2.
The time-lines were converted into data for analysis of workload by counting rounded-up ½ hours vertically and horizontally assuming the highest grade of activity in any one ½ hour. There was little grade 1 activity (7 events through out the study), which was, therefore, excluded from further consideration. The data on grade 2, 3 & 4 activity were entered into a series of Excel Workbooks for analysis allowing estimates of nightly work-load to be derived for individuals or groups of staff, at different times of the night and during phases when 5 (or more) members of staff attended the wards simultaneously. The calculations for the descriptive statistics given in the companion paper and for the SPC charts shown here were entered into the Excel work sheets.
Control charts on workload
Nightly workload at different grades of activity was plotted against time. Mean and control limits (3SDs calculated as described above) were added to each chart. Maximum capacity before and after the rota changes was estimated as the total number of staff ½ hours available between 10pm and 8am. This measure of capacity was compared with the parameters of workload defined on the control charts. Pairs of X- and R-charts were used to assess performance and judge the effects of changes to the rota. Trends in variation in nightly workload and "special cause" events (individual values lying outside 3SDs or runs of >7 values to one side of the average) were assessed using the R-chart, the mean, "common cause" and "special cause" variation using the X-chart.
Results
Phase 1: The X- and R-charts (Figure 1a&b) for overall nightly workload (grade 2,3,4 activity) both showed "special cause variation" (indicating that the mean and UCL on the X-chart was not fully reliable predictors of maximum possible work-load). The narrative in the records revealed that the "special cause" event (above UCL on X-chart) was an combination of unavoidable clinical crises but with SHOs and SpRs overlapping. The events could not have been avoided but a change in the system of managing the events might have reduced the number of staff hours required to deal with it. 12 staff spent a mean of 40.9 ½ hours per night on the wards between them with an UCL of 72.8 ½ hours. Both these values were well within the possible nightly maximum of 240 working ½ hours for 12 staff or 100 ½ hours for 5 staff. Equivalent charts for grade 3&4 activity and grade 4 activity also showed "special cause" variation on the R-chart but LCLs were now substantially negative for both sets of X-charts so that, increasingly, the charts could not provide reliable measures of performance and change (so are not shown).
Charts of workload (activity grades 2,3,4) requiring at least five members of staff simultaneously (Figures 2a&b) again showed special cause variation and the LCL on the X-chart was also negative. Staff spent an average of 2.2 ½ hours per night on such activity with an UCL of 7.9 ½ hours (possible maximum 20 ½ hours). Because it was not possible to construct valid control charts for 3&4 activity or grade 4 alone, a run chart without control limits, in which nightly workloads at different grades of activity were compared visually, is shown in Figure 3a. There was one night when, for period of 1 and a ½ hours, 5 members of staff were required simultaneously for grade 4 activity.
Given the substantial excess of capacity over all measures of workload, it was concluded, despite the lack of a full set of control charts, that the planned new rota was likely to be able to deal with the workload.
Phase 2: Compared with Phase 1, X- and R-charts (Figure 1c&d) of overall nightly work-load (grade 2,3,4 activity) showed very little change (persistence of "special cause" variation, mean 41.5 ½ hours, UCL 72.8) and there was no evidence of deterioration in performance. Similarly, for charts of 3&4 activity and 4 (not shown). For workload requiring 5 members of staff simultaneously (figure 2c&d), special cause variation had now disappeared from the R-chart, there was a reduction in the average time spent on such events from 2.2 to 1.2 ½ hours per night and the UCL had fallen from 7.9 to 5.0 ½ hours. Again the LCL was negative for this and the grade 3&4 activity and 4 only charts. Run charts without control limits are shown in Figure 3b. Compared with 3a there had been an evident reduction in the proportion of time 5 staff spent on the wards together for grade 2,3,4 and 3&4 activity but little change with respect to grade 4 alone. Just as in Phase 1 there was just one night when 5 staff were required simultaneously for grade 4 activity, this time for a period of an hour.
Conclusions
Consideration of the SPC charts suggested that change over from 12 staff on the night rota to a team of five caused little change in workload and no deterioration in performance. The overall conclusion, that the new team had the capacity to provide safe cover at night, as predicted in Phase 1, was the same as that based on the analysis shown in the companion paper. However, the use of control charts added the dimension of time to allow a visual summary of night-by-night variation in workload. This approach produced evidence of improved performance for workload requiring 5 staff simultaneously. This was important for a smaller night team because it meant that the new rota would be likely to need proportionately fewer staff simultaneously at periods of high demand. The control charts also demonstrated, far more clearly than grouped data could do, the occurrence of clinical crises making special demands on staff (five required simultaneously for emergencies). Clinical events, by their nature, will produce "special cause" variation. To really be able to predict the expected frequency of such occurrences would require very long periods of observation.
1. Wheeler, D. J. and Chambers D.S. Understanding Statistical Process Control. 1986.SPC Press, Knoxville, Tennessee
2. Wheeler D.J. Understanding variations. He key to managing chaos. 1993 SPC Press Knoxville, Tennessee.
3. Neave, H.R. "Why SPC" and "How SPC" 1994 SPC Press Knoxville, Tennessee
4. Modernisation Agency. Measurement for Improvement. 2002. www.modern.nhs.uk/improvementguides
