Papers

Effectiveness of the BBC's 999 training roadshows on cardiopulmonary resuscitation: video performance of cohort of unforewarned participants at home six months afterwards

^a Centre for Applied Public Health Medicine, University of Wales College of Medicine, Cardiff F1 8UL

Correspondence to: Dr Donnelly.

Abstract

Objective: To examine the competence of a cohort trained in cardiopulmonary resuscitation by the BBC's 999 training roadshows.
Design: Descriptive cohort study applying an innovative testing procedure to a nationwide systematic sample. The test sample received an unsolicited home visit and without warning were required to perform cardiopulmonary resuscitation on a manikin while being videoed. The videos were then analysed for effectiveness and safety using the new test.
Setting: Nine cities and surrounding areas in the United Kingdom.
Subjects: 280 people aged between 11 and 72.
Results: Thirty three (12%) trainees were able to perform effective cardiopulmonary resuscitation, but of these 14 (5%) performed one or more elements in a way that was deemed to be potentially injurious. Thus only 19 (7%) trainees were able at six months to provide safe cardiopulmonary resuscitation. In addition, large numbers of subjects failed to shout for help, effectively assess the status of the patient, or alert an ambulance. Significantly better performances were recorded by those under 45 years old (31 (14%) v 2 (4%) gave effective performances respectively, P<0.05), those who had attended a subsequent cardiopulmonary resuscitation course (8 (40%) v 25 (10%) gave effective performances respectively, P<0.0001), and those confident in their initial ability (26 (20%) v 7 (6%) gave effective performances respectively, P<0.005). Females were significantly less likely than males to perform procedures in a harmful way (117 (62%) v 10 (12%) performed safely respectively, P<0.005).
Conclusion: Television is an effective means of generating large training cohorts. Volunteers will cooperate with unsolicited testing in their home, such testing being a realistic simulation of the stress and lack of forewarning that would surround a real event. Under such conditions the performance of cardiopulmonary resuscitation was disappointing. However, retraining greatly improves performance.

Introduction

Bystander cardiopulmonary resuscitation improves survival in people who have a cardiac arrest outside hospital.¹ Consequently, increased emphasis has been placed on training the public in these techniques as a means of reducing mortality from ischaemic heart disease.^{2 3} Since 1994 the BBC has organised annual training roadshows on cardiopulmonary resuscitation throughout the United Kingdom to coincide with the broadcasting of its 999 television programme.

Although the effectiveness of bystander cardiopulmonary resuscitation has been shown, the effectiveness of mass training courses such as the BBC's 999 roadshow is less clear. Much evaluation of cardiopulmonary resuscitation training has been concerned with the performances of medical and allied professional staff rather than the lay public.^{4 5 6} With few exceptions,⁷ studies that have tried to evaluate training of the lay public have pre-warned subjects of testing either explicitly or immediately before retraining.⁸ Despite this, most results have been disappointing.⁹ We describe an innovative method of evaluating how an unforewarned lay person trained on a roadshow would perform cardiopulmonary resuscitation should a cardiac arrest occur in their home.

Subjects and methods

At each of the 10 roadshows held between April and June 1994, participants were asked to complete an optional card consenting to take part in further unspecified research. A total of 7584 course attenders completed an anonymised demographic questionnaire, and of these 6123 (81%) completed a consent card. The research entailed "cold calling" on a sample of trainees and video recording a simulated attempt at cardiopulmonary resuscitation using a Laerdal Recording Anne Manikin (Norway). Trainees from Londonderry, Northern Ireland, were excluded owing to the sensitivity at that time of undertaking unsolicited house calls there. Those who lived in a postal district outside a 32 km radius of their training centre were also excluded on logistical grounds. This left 4651 cards, from which a 6% (n = 280) stratified random sample was chosen by ordering the cards by adjacent postal district and selecting the 16th and 17th card alternately from a starting point generated by computer.

All tests were conducted between 27 and 35 weeks after the initial training. Each test was videoed for later evaluation in conjunction with the manikin recording strip. Two researchers, a man (CLlM) and a woman (see acknowledgements) visited subjects at home, without warning, and, after showing the completed consent card, asked their permission to conduct the test. The manikin and video equipment were prepared while the subject completed two questionnaires on sociodemographic data and attitudes to emergency life support. The subject then read the following passage:

"On entering a room you see a person lying still upon the floor. Please describe and demonstrate how you would approach and assess the casualty and what action you would take. I will not offer any advice or opinion other than to report the status of the casualty during your assessment."

The subject was not informed that the patient was in cardiac arrest but was expected to deduce this by assessment. The only interruption from the researchers was to confirm the status of the casualty after the appropriate assessment--that is, to report that the casualty was unconscious, not breathing, and pulseless. If subjects stated that they would phone for help they were told that help was on its way. Subjects were encouraged to continue cardiopulmonary resuscitation for a maximum of three cycles, after which they were asked when they would have stopped had this been a real life situation.

All videos and recording strips were analysed by one of us (CLlM) using guidelines developed at Cardiff. The test criteria were based on the European Resuscitation Council's guidelines,¹⁰ as these formed the core of the 999 teaching syllabus, with the categories of effective and safe representing what we considered to be reasonable deviations from these guidelines (table 1). A 20% sample was analysed by another of us (CAL) as a check on interobserver variation. All analysis was performed using SPSS for Windows.

Table 1--European Resuscitation Council's guidelines10 and study criteria for effective
and potentially injurious resuscitation
---------------------------------------------------------------------------------------------
                                                                           Potentially
                                      Guidelines         Effective         injurious
---------------------------------------------------------------------------------------------
Breathing volume (l)                     0.8-1.2            0.1-2              >21
Compression depth (mm)                    38-51             30-51              >51
Compression rate (No/min)                80-100             >/=60               NA
Ratio of breaths to compressions          2:15            1:5 (minimum)         NA
Hand position during compression    Concentrated on    Concentrated on     Any other hand
                                     centre of chest,   lower third of      position
                                     two fingers above  sternum
                                     xiphisternum
---------------------------------------------------------------------------------------------
NA = not applicable.

Results

Of the original 280 subjects, 186 (66%) took part in the study. To complete the remaining 94 tests, 136 replacements were made for the reasons listed in table 2. The final sample was comparable with the total trained cohort in terms of age, sex, and social class. Table 3 shows the kappa score of inter-rater agreement between the two assessors for the 10% sample.

Table 2--Reasons why 94 subjects needed to be replaced in study
cohort, with numbers of replacement subjects required to
complete test for each reason
----------------------------------------------------------------
                                                       No of
                                                    replacement
                                                      subjects
Reason for replacement                                (n = 136)
----------------------------------------------------------------
Refused                                                  38
Out on three occasions                                   21
Incapable owing to physical injury                        4
Possibly forewarned                                      42
Address not found, illegible, or insufficient             3
Moved with no forwarding address or outside the area     15
Tests invalid owing to:
 Equipment failure                                        7
 Prompting from another person                            6

Table 3--Agreement between raters on effectiveness of
procedures in 10% sample of cohort
--------------------------------------------------------
                              Percentage     Kappa
Procedure                     agreement     statistic
--------------------------------------------------------
Careful approach                 73           0.46
Talk to patient                  97           0.92
Shake patient                    93           0.84
Shout for help                   93       Not calculable
Clear or check airway            93           0.71
Open airway                      90           0.93
Check for breathing              87           0.68
Check for circulation            87           0.79
Telephone emergency services     70           0.68

Table 4 shows the number of subjects performing each procedure and, when appropriate, whether that procedure was effective, ineffective, or potentially injurious according to the European Resuscitation Council guidelines. Of the 210 (75%) who effectively checked for breathing, 173 did not first open the airway.

Table 4--Numbers of subjects (n = 280) performing each procedure, with type of performance when appropriate
--------------------------------------------------------------------------------------------------------------------------
                                                                                  Resuscitation performed:
                                   Resuscitation--------------------------------------------------------------------------
                                     not per-      Missing     According to
Procedure                            formed         data       guidelines10  Effectively  Ineffectively  Dangerously
--------------------------------------------------------------------------------------------------------------------------
Careful approach                       196           6              78                NA            NA            NA
Talk to patient                        180           4              96                NA            NA            NA
Shake patient                          190           4              84                NA            NA             2
Shout for help                         268           4               8                NA            NA            NA
Check airway                           242           0              38                NA            NA            NA
Open airway                            230           0              32                12             6            NA
Check for breathing                     40           0             210                NA            30            NA
Check for circulation                   69           0              25               162            24            NA
Telephone emergency services           250           0              27                NA             3            NA
Breathing volume                         9           0              68                94           101             8
Breathing interval                      21           0              80                17           162            NA
Hand position during compression        17           0              91               103            NA            69
Compression rate                        17           0              55                91           117            NA
Compression depth                       17           0              71                90            44            58
--------------------------------------------------------------------------------------------------------------------------
NA = not applicable.

Twenty seven (10%) subjects said that they would call an ambulance before beginning cardiopulmonary resuscitation, but when subjects were asked when they would stop cardiopulmonary resuscitation in a real situation 138 (49%) stated they would continue until an ambulance arrived. Of the remaining 142 (51%), 51 (36%) said that they would continue for a defined period and 55 (39%) that they would continue until the patient showed signs of revival. Sixty of these 106 (57%) subjects were asked a follow up question about what they would do when either the defined period had elapsed or the patient had failed to respond. Nineteen of them (32%) said that they would wait for the ambulance to arrive while 29 (48%) said that they would leave the patient and telephone for an ambulance after having started cardiopulmonary resuscitation.

The overall performance of cardiopulmonary resuscitation was assessed for effectiveness and safety (table 5). None performed cardiopulmonary resuscitation to the European Resuscitation Council guidelines, but 19 (7%) performed it in an effective and safe manner. A further 14 subjects (5%) were effective but performed at least one procedure that was potentially injurious. Ineffective performances were recorded for 243 subjects (87%); of these, 110 (45%) were also classed as potentially injurious. Four subjects (1.4%) did not attempt ventilation or compression.

Table 5--Overall rating of cardiopulmonary resuscitation
performed by 280 subjects
---------------------------------------------------------
Rating                                  No of subjects
---------------------------------------------------------
Conforming to guidelines10                0
Performance:
 Effective                                    19
 Effective but potentially injurious          14
 Ineffective                                 133
 Ineffective and potentially injurious       110
Resuscitation not performed                    4

Sex and social class had no influence on the effectiveness of cardiopulmonary resuscitation (table 6), but age was a significant factor, with 31 (14%) of those under 45 performing effectively compared with only two (4%) of those aged 45 or over (P<0.05). Those who felt confident before testing performed more effectively (26 out of 133, 20%) than those who were not confident or unsure (seven out of 127, 6%) (P<0.005). Subsequent training also improved performance, with eight of the 20 (40%) who had attended a subsequent course giving an effective performance against 25 of the 256 (10%) who had not. The numbers were, however, too small to make this result conclusive.

Table 6--Effectiveness and safety of performance of cardiopulmonary resuscitation by age, sex, social class, subsequent training, and
confidence before test. Values are numbers (percentages) of subjects*
----------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                 Potentially
                                           Effective         Ineffective       P value            Safe           injurious        P value
----------------------------------------------------------------------------------------------------------------------------------------------
Age (years):
 <45 (n = 219)                               31 (14)          188 (86)                          127 (58)           92 (42)
 >/=45 (n = 53)                               2 (4)            51 (96)          0.04             26 (49)           27 (51)         0.25
Sex:
 Male (n = 82)                               10 (12)           72 (88)                           34 (41)           48 (59)
 Female (n = 190)                            23 (12)          167 (88)          0.79            117 (62)           73 (38)         0.002
Social class:
 I, II, IIIN (n = 153)                       18 (12)          135 (88)                           83 (54)           70 (46)
 IIIM, IV, V (n = 98)                         9 (10)           89 (91)          0.52             55 (56)           43 (44)         0.77
Subsequent course:
 Attended (n = 20)                            8 (40)           12 (60)                           13 (65)            7 (35)
 Not attended (n = 256)                      25 (10)          231 (90)         <0.0001          140 (55)          116 (45)         0.37
Confidence before test:
 Confident (n = 133)                         26 (20)          107 (81)                           68 (51)           65 (49)
 Not confident or unsure (n = 127)            7 (6)           120 (95)          0.001            76 (60)           51 (40)         0.16
----------------------------------------------------------------------------------------------------------------------------------------------
*Four subjects did not attempt cardiopulmonary resuscitation and so are excluded from this analysis. In addition, some subjects completed the
sociodemographic questionnaire incorrectly or social class could not be coded.

There was no difference by age group or social class in those performing safe cardiopulmonary resuscitation, but only 34 (42%) men performed resuscitation safely compared with 117 (62%) women (P<0.005).

Discussion

METHODOLOGICAL ISSUES

Our methods share with others the underlying assumption that proficiency shown in performing cardiopulmonary resuscitation with a manikin is predictive of proficiency in a real situation.¹¹ To maximise the realism of the test we gave the subjects no warning and conducted all tests in their home, where over 70% of cardiac arrests occur.^{12 13 14} The presence of two observers and video equipment may have increased the anxiety of the subject and consequently enhanced or impaired performance, although the extent to which this compares with the stress of a genuine emergency is impossible to measure.

Although cardiopulmonary resuscitation from a bystander improves outcome, what constitutes effective cardiopulmonary resuscitation is unproved. Cardiopulmonary resuscitation judged retrospectively to be poor may still be associated with improved outcome, although this may be dependent on a rapid response by the emergency medical services.^{12 13 14 15} Several studies determining the effect of the quality of cardiopulmonary resuscitation on outcome have considered effective cardiopulmonary resuscitation to be that which is observed to produce a visible expansion of the chest and a palpable carotid or femoral pulse.^{16 17} The minimum prerequisite to achieve this is not known, however. As a result, the test criteria must be somewhat arbitrary with the aim of being liberal, such that any performance that could reasonably be considered beneficial will be classed as effective.

POSSIBLE PROBLEMS WITH INSTRUCTION

As there is no record of the degree of competency achieved immediately after training, we could not determine whether our results are due to inadequate instruction, poor retention, or a combination of both. Several comparative studies have shown the time since training to be closely associated with a decrease in proficiency.^{5 18} As reported by others, this study indicates that some form of regular retraining is required to maintain adequate skills in cardiopulmonary resuscitation, although those willing to attend retraining courses may be more motivated than those who do not, such motivation possibly affecting initial learning or retention.^{8 19}

The organisation of the courses may also have limited the initial absorption of techniques. The structure of the roadshows was similar to the Save A Life campaign, which has been criticised for its short training time (2.5 hours) and the fact that instructors come from many different organisations.²⁰ Retention of cardiopulmonary resuscitation skills was greater among those who attended an eight hour rather than a four hour course.²¹ Given the breadth of the syllabus, the 2.5 hours given to the roadshows may be less than required. The provision of training by many organisations may be problematic as different bodies have different training agendas. Anecdotal evidence collected during testing suggested that predefined methods were replaced by techniques idiosyncratic to particular organisations. This caused confusion among trainees, who had been shown a video of the approved procedures, and any variation from the approved techniques may not have helped reinforcement. Some instructors also introduced entirely new components, such as cardiopulmonary resuscitation performed by two people, which took up teaching time and confused trainees. Similar observations were noted in the study of Kaye et al, who found that trainees performed poorly on manikins after teaching sessions on cardiopulmonary resuscitation.²² The BBC has taken measures to rectify this in its current roadshows.

ACTIVATING THE EMERGENCY SERVICES

The most important aspect of emergency life support is activating the emergency services, but, surprisingly, only 27 (10%) subjects did this before initiating cardiopulmonary resuscitation. Although many subjects simply assumed that an ambulance was activated, a sizeable minority either did not mention the emergency services or stated that they would send for an ambulance at an inappropriate stage. An American study found that people who attempt cardiopulmonary resuscitation are more likely to telephone for an ambulance.¹⁴ However, a local population survey showed that on encountering someone in cardiac arrest, 65% of subjects would firstly phone for an ambulance and this increased to 73% among those who had received no formal training in emergency life support.²³ With only ambiguous evidence further study is required to determine whether a false sense of empowerment among trainees may mean that they delay activating the emergency services.

In our study cardiopulmonary resuscitation was initiated by 90 (32%) subjects who did not first establish that there was no pulse; 36 of them (40%) performed at least one potentially injurious procedure. In one study only 20% of medical students checked the pulse effectively²⁴, whereas in another 70% of lay people established whether there was a pulse.²⁵ As discussed by others, this may be an artefact of the testing method: subjects are anxious to offer treatment before establishing a diagnosis, which may not apply in a real situation.²⁶ Over nine years in Seattle only one case was documented in which cardiopulmonary resuscitation was inappropriately performed, and on this occasion no harm was caused.²⁷

What constitutes injurious cardiopulmonary resuscitation is arbitrary. Even cardiopulmonary resuscitation considered to be performed correctly may be associated with a wide range of complications.²⁸ In all, 124 subjects (44%) performed one or more potentially injurious procedures. However, any potential injurious effect must be balanced against the certainty of harm caused by withholding appropriate cardiopulmonary resuscitation.

CONCLUSIONS

We believe that the unforewarned video testing that we used during this study provides a valuable tool for assessing cardiopulmonary resuscitation skills and has wider applications for research in this field. It successfully simulates the stress and lack of warning that would accompany a real event. We have shown that volunteers will cooperate with cold call testing, but under these conditions the performance of cardiopulmonary resuscitation is disappointing; these results are similar to those from studies of medical and nursing staff.^{4 5 6 29}

Thus, the 999 roadshow is undeniably successful at recruiting large cohorts of volunteers, but attention must now turn to improving the standardisation and quality of instruction and to developing retraining strategies.³⁰ Without such developments the unique opportunities offered by recruitment through television will be lost. These poor results may also indicate a need to simplify resuscitation protocols.

We thank Tristan Vaughan, Sarajane Doonan, Carla Guisti, Kirsty Ogilvie, and Christine Verrall for help in collecting data; Stephen Leaves for advice on evaluation; and Julie Harrison for entering data. We also thank Andrea Wills, executive producer, 999, and the 999 team for their cooperation during this study.

Funding: British Heart Foundation.

Conflict of interest: None.

1. Cummins RO, Eisenberg MS. Pre-hospital cardiopulmonary resuscitation--is it effective? JAMA 1985;253:2408-12.
2. Cummins RO, Ornato JP, Thies WH, Pepe P. Improving survival from sudden cardiac arrest: "the chain of survival" concept. Circulation 1991;83:1833-47.
3. Baskett PJF. The need to disseminate knowledge of resuscitation into the community. Anaesthaesia 1985;37:74-6.
4. Gass DA, Curry L. Physicians' and nurses' retention of knowledge and skills after training in cardiopulmonary resuscitation. Can Med Assoc J 1983;128:550-1. [Abstract]
5. Fossel M, Kiskaddon RT, Sternbach GL. Retention of CPR skill by medical students. J Med Educ 1983;58:568-75. [Medline]
6. Deliere DM, Schneider MPH. A study of cardiopulmonary resuscitation technical skill retention among trained EMT-A's. EMT Journal 1980;4:57-60.
7. Kortilla K, Vertio H, Savolainen K. Importance of using proper techniques to teach cardiopulmonary resuscitation to laymen. Acta Anaesthesiol Scand 1979;23:235-41. [Medline]
8. Kalmthout P, Speth P, Rutten J, Vonk J. Evaluation of lay skills in cardiopulmonary resuscitation. Br Heart J 1985;53:562-6. [Abstract/Free Full Text]
9. Moser DK, Coleman S. Recommendations for improving cardiopulmonary resuscitation skills retention. Heart Lung 1992;4:372-80.
10. European Resuscitation Council. Guidelines for basic life support. Resuscitation 1992;24:103-10. [Medline]
11. Cobb LA, Hallstrom PA. Community based cardiopulmonary resuscitation: what have we learned? Ann N Y Acad Sci 1982;382:330-42.
12. Hearne TR, Cummins RO. Improved survival from cardiac arrest in the community. Pacing Clin Electrophysiol 1988;2:1968-73.
13. Litwin PE, Eisenberg MS, Hallstrom AP, Cummins RO. The location of collapse and its effect on survival from cardiac arrest. Ann Emerg Med 1987;16:787-91. [Medline]
14. Weston CFM, Hughes DW, Donnelly MDI. Potential impact upon community mortality rates of training citizens in cardiopulmonary resuscitation. J R Coll Physicians Lond 1994;28:402-6. [Medline]
15. Kowalski R, Thompson BM, Horwitz L, Stueven H, Aprahamian C, Darin JC. Bystander CPR in prehospital coarse ventricular fibrillation. Ann Emerg Med 1984;13:1016-20. [Medline]
16. Gallagher EJ, Lombardi G, Gennis P. Effectiveness of bystander cardiopulmonary resuscitation and survival following out-of-hospital cardiac arrest. JAMA 1995;274:1922-5. [Abstract/Free Full Text]
17. Wik L, Steen PA, Bircher NG. Quality of bystander cardiopulmonary resuscitation influences outcome after prehospital cardiac arrest. Resuscitation 1994;28:195-203. [Medline]
18. McKenna SP, Glendon AI. Occupational first aid training: decay in cardiopulmonary resuscitation (CPR) skills. Journal of Occupational Psychology 1985;58:109-17.
19. Cobb LA, Hallstrom AP, Thompson RG, Mandel LP, Copass MK. Community cardiopulmonary resuscitation. Ann Rev Med 980;31:453-62. [Medline]
20. Hughes RJV, Aldridge BJ. Do resuscitation schemes save lives? Health Education Journal 1988;47:53-5.
21. Gombeski WR; Effron DM, Ramirez AG, Moore TJ. Impact on retention: comparison of two CPR training programs. Am J Public Health 1982;72:849-52. [Abstract/Free Full Text]
22. Kaye W, Rallis RF, Mancini ME. The problem of poor retention of cardiopulmonary resuscitation skills may lie with the instructor, not the learner or the curriculum. Resuscitation 1991;21:67-87. [Medline]
23. Assar D. Should South Glamorgan Health Authority invest in a community cardiopulmonary resuscitation project?. Pontypridd: University of Glamorgan, 1995:41.
24. Fossel M, Kiskaddon RT, Sternbach GL. Retention of CPR skills by medical students. J Med Educ 1983;58:568-75.
25. Weaver FJ, Ramirez AG, Dorfman SB, Raizner AE. Trainees' retention of cardiopulmonary resuscitation: how quickly do they forget? JAMA 1979;24:901-3.
26. Martin WJ, Loomis JH, Lloyd CW. CPR skills: achievement and retention under stringent and relaxed criteria. Am J Public Health 1983;73:1310-2. [Abstract/Free Full Text]
27. Cummins RO, Eisenburg MS. CPR--American style. BMJ 1985;291:1401-3.
28. Jones J, Fletter B. Complications after cardiopulmonary resuscitation. Am J Public Health 1994;12:687-8.
29. David J, Prior-Willeard PF. Resuscitation skills of MRCP candidates. BMJ 1993;306:1578-9.
30. Berden HJJM, Willhelms FF, Hendrick JMA, Pijls NH, Knape JTA. How frequently should basic cardiopulmonary resuscitation training be repeated to maintain adequate skills? BMJ 1993;306:1576-7.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

This article has been cited by other articles:

• Sim, M S, Jo, I J, Song, H G (2009). Basic cardiac life support education for non-medical hospital employees. Emerg. Med. J. 26: 327-330 [Abstract] [Full text]  
• Chamberlain, D. A., Hazinski, M. F., On behalf of the European Resuscitation Council, , the American Heart Association, , the Heart and Stroke Foundation of Canada, , the Resuscitation Council of Southern Africa, , the Australia and New Zealand Resuscitation Counci, , the Consejo Latino-Americano de Resusucitacion, (2003). Education in Resuscitation: An ILCOR Symposium: Utstein Abbey: Stavanger, Norway: June 22-24, 2001. Circulation 108: 2575-2594 [Full text]  
• Woollard, M. (2002). Bringing order from chaos: The challenge of training lay responders to use automated external defibrillators and provide cardiopulmonary resuscitation. Health Education Journal 61: 197-211 [Abstract]  
• Ewy, G. A., Ornato, J. P. (2000). Emergency cardiac care: introduction. J Am Coll Cardiol 35: 825-880 [Full text]  
• Wills, A. (1997). Having some lifesaving skills must be better than having none. BMJ 314: 222-222 [Full text]