Papers

Effectiveness of the BBC's 999 training roadshows on cardiopulmonary resuscitation: video performance of cohort of unforewarned participants at home six months afterwards

BMJ 1996; 313 doi: https://doi.org/10.1136/bmj.313.7062.912 (Published 12 October 1996) Cite this as: BMJ 1996;313:912
  1. C Ll Morgan, research officera,
  2. P D Donnelly, senior lecturera,
  3. C A Lester, research officera,
  4. D H Assar, project managera
  1. a Centre for Applied Public Health Medicine, University of Wales College of Medicine, Cardiff F1 8UL
  1. Correspondence to: Dr Donnelly.
  • Accepted 6 September 1996

Abstract

Objective: To examine the competence of a cohort trained in cardiopulmonary resuscitation by the BBC's 999 training roadshows.

Design: Descriptive cohort study applying an innovative testing procedure to a nationwide systematic sample. The test sample received an unsolicited home visit and without warning were required to perform cardiopulmonary resuscitation on a manikin while being videoed. The videos were then analysed for effectiveness and safety using the new test.

Setting: Nine cities and surrounding areas in the United Kingdom.

Subjects: 280 people aged between 11 and 72.

Results: Thirty three (12%) trainees were able to perform effective cardiopulmonary resuscitation, but of these 14 (5%) performed one or more elements in a way that was deemed to be potentially injurious. Thus only 19 (7%) trainees were able at six months to provide safe cardiopulmonary resuscitation. In addition, large numbers of subjects failed to shout for help, effectively assess the status of the patient, or alert an ambulance. Significantly better performances were recorded by those under 45 years old (31 (14%) v 2 (4%) gave effective performances respectively, P<0.05), those who had attended a subsequent cardiopulmonary resuscitation course (8 (40%) v 25 (10%) gave effective performances respectively, P<0.0001), and those confident in their initial ability (26 (20%) v 7 (6%) gave effective performances respectively, P<0.005). Females were significantly less likely than males to perform procedures in a harmful way (117 (62%) v 10 (12%) performed safely respectively, P<0.005).

Conclusion: Television is an effective means of generating large training cohorts. Volunteers will cooperate with unsolicited testing in their home, such testing being a realistic simulation of the stress and lack of forewarning that would surround a real event. Under such conditions the performance of cardiopulmonary resuscitation was disappointing. However, retraining greatly improves performance.

Key messages

  • Training for the lay public in cardiopulmonary resuscitation has increased with such initiatives as BBC television's 999 national roadshows, but the effectiveness of this training has not been rigorously evaluated

  • In this study 280 people who had attended a roadshow were tested, unforewarned, six months later in their home, their management of a simulated case of cardiac arrest being videotaped for later analysis

  • Only 12% of subjects performed cardiopulmo- nary resuscitation effectively

  • As well as performing cardiopulmonary resusci- tation ineffectively, 39% of subjects performed one or more procedures in a way that could complicate the recovery of a casualty

  • Although the 999 roadshows undoubtedly recruit many lay people, attention should now be given to retraining strategies

Introduction

Bystander cardiopulmonary resuscitation improves survival in people who have a cardiac arrest outside hospital.1 Consequently, increased emphasis has been placed on training the public in these techniques as a means of reducing mortality from ischaemic heart disease.2 3 Since 1994 the BBC has organised annual training roadshows on cardiopulmonary resuscitation throughout the United Kingdom to coincide with the broadcasting of its 999 television programme.

Although the effectiveness of bystander cardiopulmonary resuscitation has been shown, the effectiveness of mass training courses such as the BBC's 999 roadshow is less clear. Much evaluation of cardiopulmonary resuscitation training has been concerned with the performances of medical and allied professional staff rather than the lay public.4 5 6 With few exceptions,7 studies that have tried to evaluate training of the lay public have pre-warned subjects of testing either explicitly or immediately before retraining.8 Despite this, most results have been disappointing.9 We describe an innovative method of evaluating how an unforewarned lay person trained on a roadshow would perform cardiopulmonary resuscitation should a cardiac arrest occur in their home.

Subjects and methods

At each of the 10 roadshows held between April and June 1994, participants were asked to complete an optional card consenting to take part in further unspecified research. A total of 7584 course attenders completed an anonymised demographic questionnaire, and of these 6123 (81%) completed a consent card. The research entailed “cold calling” on a sample of trainees and video recording a simulated attempt at cardiopulmonary resuscitation using a Laerdal Recording Anne Manikin (Norway). Trainees from Londonderry, Northern Ireland, were excluded owing to the sensitivity at that time of undertaking unsolicited house calls there. Those who lived in a postal district outside a 32 km radius of their training centre were also excluded on logistical grounds. This left 4651 cards, from which a 6% (n = 280) stratified random sample was chosen by ordering the cards by adjacent postal district and selecting the 16th and 17th card alternately from a starting point generated by computer.

All tests were conducted between 27 and 35 weeks after the initial training. Each test was videoed for later evaluation in conjunction with the manikin recording strip. Two researchers, a man (CLlM) and a woman (see acknowledgements) visited subjects at home, without warning, and, after showing the completed consent card, asked their permission to conduct the test. The manikin and video equipment were prepared while the subject completed two questionnaires on sociodemographic data and attitudes to emergency life support. The subject then read the following passage:

“On entering a room you see a person lying still upon the floor. Please describe and demonstrate how you would approach and assess the casualty and what action you would take. I will not offer any advice or opinion other than to report the status of the casualty during your assessment.”

The subject was not informed that the patient was in cardiac arrest but was expected to deduce this by assessment. The only interruption from the researchers was to confirm the status of the casualty after the appropriate assessment—that is, to report that the casualty was unconscious, not breathing, and pulseless. If subjects stated that they would phone for help they were told that help was on its way. Subjects were encouraged to continue cardiopulmonary resuscitation for a maximum of three cycles, after which they were asked when they would have stopped had this been a real life situation.

All videos and recording strips were analysed by one of us (CLlM) using guidelines developed at Cardiff. The test criteria were based on the European Resuscitation Council's guidelines,10 as these formed the core of the 999 teaching syllabus, with the categories of effective and safe representing what we considered to be reasonable deviations from these guidelines (table 1). A 20% sample was analysed by another of us (CAL) as a check on interobserver variation. All analysis was performed using SPSS for Windows.

Table 1

European Resuscitation Council's guidelines10 and study criteria for effective and potentially injurious resuscitation

View this table:

Results

Of the original 280 subjects, 186 (66%) took part in the study. To complete the remaining 94 tests, 136 replacements were made for the reasons listed in table 2. The final sample was comparable with the total trained cohort in terms of age, sex, and social class. Table 3 shows the kappa score of inter-rater agreement between the two assessors for the 10% sample.

Table 2

Reasons why 94 subjects needed to be replaced in study cohort, with numbers of replacement subjects required to complete test for each reason

View this table:
Table 3

Agreement between raters on effectiveness of procedures in 10% sample of cohort

View this table:

Table 4 shows the number of subjects performing each procedure and, when appropriate, whether that procedure was effective, ineffective, or potentially injurious according to the European Resuscitation Council guidelines. Of the 210 (75%) who effectively checked for breathing, 173 did not first open the airway.

Table 4

Numbers of subjects (n = 280) performing each procedure, with type of performance when appropriate

View this table:

Twenty seven (10%) subjects said that they would call an ambulance before beginning cardiopulmonary resuscitation, but when subjects were asked when they would stop cardiopulmonary resuscitation in a real situation 138 (49%) stated they would continue until an ambulance arrived. Of the remaining 142 (51%), 51 (36%) said that they would continue for a defined period and 55 (39%) that they would continue until the patient showed signs of revival. Sixty of these 106 (57%) subjects were asked a follow up question about what they would do when either the defined period had elapsed or the patient had failed to respond. Nineteen of them (32%) said that they would wait for the ambulance to arrive while 29 (48%) said that they would leave the patient and telephone for an ambulance after having started cardiopulmonary resuscitation.

The overall performance of cardiopulmonary resuscitation was assessed for effectiveness and safety (table 5). None performed cardiopulmonary resuscitation to the European Resuscitation Council guidelines, but 19 (7%) performed it in an effective and safe manner. A further 14 subjects (5%) were effective but performed at least one procedure that was potentially injurious. Ineffective performances were recorded for 243 subjects (87%); of these, 110 (45%) were also classed as potentially injurious. Four subjects (1.4%) did not attempt ventilation or compression.

Table 5

Overall rating of cardiopulmonary resuscitation performed by 280 subjects

View this table:

Sex and social class had no influence on the effectiveness of cardiopulmonary resuscitation (table 6), but age was a significant factor, with 31 (14%) of those under 45 performing effectively compared with only two (4%) of those aged 45 or over (P<0.05). Those who felt confident before testing performed more effectively (26 out of 133, 20%) than those who were not confident or unsure (seven out of 127, 6%) (P<0.005). Subsequent training also improved performance, with eight of the 20 (40%) who had attended a subsequent course giving an effective performance against 25 of the 256 (10%) who had not. The numbers were, however, too small to make this result conclusive.

Table 6

Effectiveness and safety of performance of cardiopulmonary resuscitation by age, sex, social class, subsequent training, and confidence before test. Values are numbers (percentages) of subjects*

View this table:

There was no difference by age group or social class in those performing safe cardiopulmonary resuscitation, but only 34 (42%) men performed resuscitation safely compared with 117 (62%) women (P<0.005).

Discussion

METHODOLOGICAL ISSUES

Our methods share with others the underlying assumption that proficiency shown in performing cardiopulmonary resuscitation with a manikin is predictive of proficiency in a real situation.11 To maximise the realism of the test we gave the subjects no warning and conducted all tests in their home, where over 70% of cardiac arrests occur.12 13 14 The presence of two observers and video equipment may have increased the anxiety of the subject and consequently enhanced or impaired performance, although the extent to which this compares with the stress of a genuine emergency is impossible to measure.

Although cardiopulmonary resuscitation from a bystander improves outcome, what constitutes effective cardiopulmonary resuscitation is unproved. Cardiopulmonary resuscitation judged retrospectively to be poor may still be associated with improved outcome, although this may be dependent on a rapid response by the emergency medical services.12 13 14 15 Several studies determining the effect of the quality of cardiopulmonary resuscitation on outcome have considered effective cardiopulmonary resuscitation to be that which is observed to produce a visible expansion of the chest and a palpable carotid or femoral pulse.16 17 The minimum prerequisite to achieve this is not known, however. As a result, the test criteria must be somewhat arbitrary with the aim of being liberal, such that any performance that could reasonably be considered beneficial will be classed as effective.

POSSIBLE PROBLEMS WITH INSTRUCTION

As there is no record of the degree of competency achieved immediately after training, we could not determine whether our results are due to inadequate instruction, poor retention, or a combination of both. Several comparative studies have shown the time since training to be closely associated with a decrease in proficiency.5 18 As reported by others, this study indicates that some form of regular retraining is required to maintain adequate skills in cardiopulmonary resuscitation, although those willing to attend retraining courses may be more motivated than those who do not, such motivation possibly affecting initial learning or retention.8 19

The organisation of the courses may also have limited the initial absorption of techniques. The structure of the roadshows was similar to the Save A Life campaign, which has been criticised for its short training time (2.5 hours) and the fact that instructors come from many different organisations.20 Retention of cardiopulmonary resuscitation skills was greater among those who attended an eight hour rather than a four hour course.21 Given the breadth of the syllabus, the 2.5 hours given to the roadshows may be less than required. The provision of training by many organisations may be problematic as different bodies have different training agendas. Anecdotal evidence collected during testing suggested that predefined methods were replaced by techniques idiosyncratic to particular organisations. This caused confusion among trainees, who had been shown a video of the approved procedures, and any variation from the approved techniques may not have helped reinforcement. Some instructors also introduced entirely new components, such as cardiopulmonary resuscitation performed by two people, which took up teaching time and confused trainees. Similar observations were noted in the study of Kaye et al, who found that trainees performed poorly on manikins after teaching sessions on cardiopulmonary resuscitation.22 The BBC has taken measures to rectify this in its current roadshows.

ACTIVATING THE EMERGENCY SERVICES

The most important aspect of emergency life support is activating the emergency services, but, surprisingly, only 27 (10%) subjects did this before initiating cardiopulmonary resuscitation. Although many subjects simply assumed that an ambulance was activated, a sizeable minority either did not mention the emergency services or stated that they would send for an ambulance at an inappropriate stage. An American study found that people who attempt cardiopulmonary resuscitation are more likely to telephone for an ambulance.14 However, a local population survey showed that on encountering someone in cardiac arrest, 65% of subjects would firstly phone for an ambulance and this increased to 73% among those who had received no formal training in emergency life support.23 With only ambiguous evidence further study is required to determine whether a false sense of empowerment among trainees may mean that they delay activating the emergency services.

In our study cardiopulmonary resuscitation was initiated by 90 (32%) subjects who did not first establish that there was no pulse; 36 of them (40%) performed at least one potentially injurious procedure. In one study only 20% of medical students checked the pulse effectively24, whereas in another 70% of lay people established whether there was a pulse.25 As discussed by others, this may be an artefact of the testing method: subjects are anxious to offer treatment before establishing a diagnosis, which may not apply in a real situation.26 Over nine years in Seattle only one case was documented in which cardiopulmonary resuscitation was inappropriately performed, and on this occasion no harm was caused.27

What constitutes injurious cardiopulmonary resuscitation is arbitrary. Even cardiopulmonary resuscitation considered to be performed correctly may be associated with a wide range of complications.28 In all, 124 subjects (44%) performed one or more potentially injurious procedures. However, any potential injurious effect must be balanced against the certainty of harm caused by withholding appropriate cardiopulmonary resuscitation.

CONCLUSIONS

We believe that the unforewarned video testing that we used during this study provides a valuable tool for assessing cardiopulmonary resuscitation skills and has wider applications for research in this field. It successfully simulates the stress and lack of warning that would accompany a real event. We have shown that volunteers will cooperate with cold call testing, but under these conditions the performance of cardiopulmonary resuscitation is disappointing; these results are similar to those from studies of medical and nursing staff.4 5 6 29

Thus, the 999 roadshow is undeniably successful at recruiting large cohorts of volunteers, but attention must now turn to improving the standardisation and quality of instruction and to developing retraining strategies.30 Without such developments the unique opportunities offered by recruitment through television will be lost. These poor results may also indicate a need to simplify resuscitation protocols.

We thank Tristan Vaughan, Sarajane Doonan, Carla Guisti, Kirsty Ogilvie, and Christine Verrall for help in collecting data; Stephen Leaves for advice on evaluation; and Julie Harrison for entering data. We also thank Andrea Wills, executive producer, 999, and the 999 team for their cooperation during this study.

Footnotes

  • Funding British Heart Foundation.

  • Conflict of interest None.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
View Abstract