Papers

Trainee satisfaction before and after the Calman reforms of specialist training: questionnaire survey

Elisabeth Paice, dean director,  Maryanne Aitken, project manager,  George Cowan, postgraduate dean,  Shelley Heard, postgraduate dean. 

North Thames Department of Postgraduate Medical and Dental Education, London WC1N 3EJ

Correspondence to: E Paice epaice{at}tpmde.ac.uk

	Abstract

Top Abstract Introduction Participants and methods Results Discussion References

Objectives: To evaluate the impact of the Calman reforms of higher specialist training on trainee satisfaction.
Design: Questionnaire surveys using portable electronic survey units, two years apart.
Setting: Postgraduate, teaching, district general, and community NHS trusts in North Thames. North Thames deanery includes London north of the Thames, Essex, and Hertfordshire.
Participants: Trainees in all grades and all specialties: 3078 took part in the first survey and 3517 in the second survey.
Main outcome measures: Trainees' satisfaction with training in their current post, including educational objectives, training agreements, induction, consultant feedback, hands on experience acquired, use of log books, consultant supervision, and overall satisfaction with the post.
Results: In the second survey respondents were more likely to have discussed educational objectives with their consultant, used a log book, and had useful feedback from their consultant. They were more likely to give high ratings to induction, consultant supervision, and hands on experience acquired in the post. Each of these elements was associated with increased satisfaction with the post overall. Improvements were most noticeable at the level of specialist registrar, but changes in the same direction were also seen in more junior grades.
Conclusions: After the reforms of specialist training, trainees in all grades reported greater satisfaction with their current posts. The changes required extra training time and effort from consultants.

	Introduction

Top Abstract Introduction Participants and methods Results Discussion References

In 1992 Kenneth Calman, then chief medical officer, set up a working group to bring the British system of specialist training into line with the requirements of the European medical directives. The resulting report recommended combining the registrar and senior registrar grades into a unified specialist registrar grade and defining the curriculum and minimum training period for each specialty, the successful completion of which would lead to admission to the specialist register.¹ Features of the new system were set out in A Guide to Specialist Registrar Training and included educational objective setting, training agreements, and induction at the start of each placement; rotational placements designed to offer specified experience; and regular feedback on progress from the supervising consultant.² The reforms were to be cost neutral, and no additional resources were made available to NHS trusts for their implementation. Transition to the new system began in December 1995 and was completed in April 1997.

The new arrangements placed more emphasis on structured teaching and supervised learning and less on experiential apprenticeship.³ Although the reforms received a cautious welcome,^4-6 there were anxieties about the impact of shortening the training time on trainees' experience⁷ especially since junior doctors' hours were being reduced.⁸ It was expected that the reforms would lead to fewer trainees and more consultants.⁹ Consultants feared they would be expected to take on extra responsibility for out of hours emergency work, including resident on-call, ^{10 11} although this was denied by senior doctors involved in planning the reforms. ^{12 13} The implementation of the reforms seemed likely to create a major new training workload for consultants.¹⁴ At the same time they were losing control over appointment of their own junior colleagues, who were to be appointed to regional programmes and allocated to posts by a regional committee. It was not clear whether the reforms could be delivered without additional resources, how motivated the consultants would be to implement change, whether higher specialist trainees would consider their training improved, or what the impact would be on the training of more junior grades.

To evaluate the impact of the reforms on specialist registrars (including old style registrars and senior registrars) and any knock-on effects on more junior grades (preregistration house officers and senior house officers), we planned two surveys of trainees in all grades and all specialties in our region: one during transition to the new system and one two years later.

	Participants and methods

Top Abstract Introduction Participants and methods Results Discussion References

Surveys
The first survey took place during transition in November and December 1996, with postal questionnaires to non-responders in January 1997. The second survey took place 18 months after transition was completed in November and December 1998, with postal questionnaires to non-responders in January 1999. Although some trainees would have been surveyed twice, it was unlikely any would have been in the same post two years later. The questions were developed from those used for monitoring senior house officer training,¹⁵ modified to be appropriate for all grades. The questionnaire was loaded on to portable electronic survey units. Each question was displayed on a screen, and the participant keyed in the number of the response chosen. Confidentiality was assured. Each trust was provided with from one to five survey units for one to two weeks. A contact person (usually the postgraduate centre manager) organised the survey locally by publicising the survey to trainees, receiving the units and ensuring they were accessible to the trainees, preparing a list of trainees, and making a note of those who responded or those who were unavailable to respond (for example, on leave, moved on). Non-responders were chased up by postal questionnaires.

Participants
Our study sample was defined as all trainees available to respond at the time of the survey in participating NHS trusts in North Thames. We excluded doctors not in training grades and locums who had been in post less than two weeks. We made no attempt to contact trainees not based in participating trustsfor example, those in public health, industry, hospices, private hospitals, or general practice.

Statistical analysis
We used SPSS software (version 8.0) for our analyses. We calculated significance with the Mann-Whitney U test for ordinal data and the ² test for categorical data.

	Results

Top Abstract Introduction Participants and methods Results Discussion References

Respondents
Fifty nine of 61 NHS trusts agreed to take part in the first survey. Of the 4250 trainees available, 3078 responded (response rate 72.4%). All trusts agreed to take part in the second survey. Of the 4765 trainees available, 3517 responded (73.8%). The table details the characteristics of the respondents. The distribution of respondents by grade and specialty did not differ importantly from that of the postgraduate dean's database for the year concerned. The larger numbers in the second survey reflected the participation of two more trusts, a longer duration of the survey in the bigger trusts, and a higher response rate. Not all respondents answered every question.

Training

Educational objectives and training agreements
A Guide to Specialist Registrar Training states that at the beginning of each placement the trainer and trainee should discuss educational objectives. Trainees in the second survey were more likely to report having had such a discussion with their consultant (fig 1a). The specialty most compliant with this requirement was accident and emergency medicine (92% of specialist registrars in the first survey and 97% in the second) whereas ophthalmology was the least compliant (28% v 44%). Compliance increased from 37% to 46% in the surgical specialties and from 42% to 63% in the medical specialties. Compliance increased in all types of NHS trust, the most noticeable change being in postgraduate institutions (38% to 59%) and the least noticeable in the already compliant community trusts (75% to 77%). The training guide also recommends that an individual training agreement be signed by trainee and trainer. Evidence of compliance was disappointing, despite a requirement in the educational contract between trusts and the postgraduate deans for this to be a feature of all training posts. Compliance increased from 18% to 29% among specialist registrars and from 19% to 29% among junior grades.

View larger version (58K): [in this window] [in a new window]   Trainees' ratings of aspects of Calman reforms

Induction
The training guide recommends specialist registrars have an induction to each post in a training programme. The postgraduate deans have been encouraging good induction arrangements for junior grades for several years. Since the quality of the induction determines its usefulness, we used a rating scale to measure satisfaction (fig 1b). Trainees at all levels in the second survey gave higher ratings to the induction they received. Obstetrics and gynaecology showed the most improvement, with ratings of "very poor" or "poor" decreasing from 38% to 17% for specialist registrars and 39% to 26% for junior grades. Similar improvements were seen in all types of trusts.

Consultants' feedback
The training guide recommends regular informal discussions between the trainee and the supervising consultant about the trainee's progress. Consultants' feedback of this sort for all grades had been recommended by the postgraduate deans for years, and training in appraisal skills had been provided by them from 1993. Trainees in the second survey were more likely to have had useful feedback or to know that feedback was planned (fig 1c). Improvement was seen across all the specialties. At specialist registrar level trainees in all types of trust were more likely to have had useful feedback in the second survey, with the biggest improvements being in the teaching (38% to 50%) and postgraduate (28% to 40%) hospitals. For junior grades improvement was most noticeable in district general hospital (31% to 40%) or community (47% to 56%) posts.

Hands-on experience
One of the aims of structured training was to ensure that all trainees acquired the experience required by the specialist curriculum. We asked trainees to rate the experience they were acquiring in their current post. There was a slight but significant (Mann-Whitney U test 949 550, P<0.001) improvement in ratings for specialist registrars and no significant change for junior grades. No significant difference was found between specialties or type of trust.

Log books
Although there is no requirement in the training guide to introduce a log book, many specialties have developed these as a means of recording progress in gaining experience required by the specialist curriculum. Some royal colleges also issue log books to senior house officers. We found a noticeable increase between the surveys in the number of trainees in all grades and specialties who had used a log book (fig 1d). Usage among specialist registrars increased in obstetrics and gynaecology from 27% to 69%, in medical specialties from 15% to 55%, and in anaesthetics from 48% to 93%. The surgical specialties showed the smallest increase (80% to 86%). The greatest use among senior house officers was in anaesthetics (79% to 91%), and the lowest use was in accident and emergency medicine (11% to 17%). No significant difference in use of log books was found between the different types of training trust.

Supervision by consultants
Trainees' ratings of the quality of supervision by consultants improved between surveys (fig 1e). The biggest improvements were seen in medical specialties and anaesthetics. For specialist registrars the major improvement was seen in teaching hospitals and postgraduate institutes, whereas for junior grades the improvement was more noticeable in district general hospitals.

Overall satisfaction
Overall satisfaction with the post was measured by asking the respondent, "How would you describe this post to a friend who was thinking of applying for it?" The responses were on a scale of 1 to 5 where 1 was "very poor" and 5 was "excellent" (fig 1f) and showed a modest but significant improvement. The changes were in the same direction in all grades, specialties, and types of NHS trust.

	Discussion

Top Abstract Introduction Participants and methods Results Discussion References

After the introduction of the Calman reforms of specialist training, specialist registrars were more likely to report satisfaction with key elements of their training. Similar, if less noticeable, improvements were noted in more junior grades.

Strengths and weaknesses
StrengthsThe study included trainees from all grades, specialties, and types of NHS trust, in the largest deanery in the United Kingdom where 25% of the country's specialist registrars are trained. The two surveys were carried out at the same time of year, reducing bias from seasonal variations in workload or casemix. The technology provided assurance of confidentiality, encouraged a good response rate, and reduced risk of errors in the transfer of data.

Meaning and implications
The satisfaction of specialist registrars with training within their current post increased after the implementation of the Calman reforms. The reforms did not seem to have diverted attention away from the needs of more junior grades and may have had a positive effect on their training. There was no evidence that higher specialist trainees thought they were acquiring less experience in their posts or that job satisfaction had decreased, despite expectations to the contrary. ^{17 18} The reforms of specialist training did not take place in isolation but were part of a general trend towards educational improvement and better working conditions.¹⁹ Much of this improvement required additional efforts by consultants both in delivering training and organising training programmes. The impact of this extra work has undoubtedly been to load additional stresses on to an already overburdened consultant workforce. ^{20 21}

Further work
Further work should address the management of the training programmes including recruitment, rotations, and the annual record of in-service training assessments. As the training period becomes shorter, fitness to take on the consultant role at completion of training should be evaluated. Further evaluation of these educational reforms regarding consultant time, effort, and stress are urgently needed, especially as the scale of consultant expansion on which the reforms were predicated has not so far materialised.

	Acknowledgments

We thank Ray Flux of CivilEyes for advice and for supplying, delivering, and downloading the survey units; the postgraduate centre managers and medical personnel officers for ensuring a good response rate; the trainees; and the consultant trainers.

Contributors: EP conceived the project, piloted the first version of the questionnaire, analysed the results, and drafted the paper; she will act as guarantor for the paper. MA project managed the second survey, entered the postal data, and checked the analysis. GC and SH contributed to the design of the project, the development of the questionnaire, and the writing of the final manuscript.

	Footnotes

Funding: None.

Competing interests: None declared.

	References

Top Abstract Introduction Participants and methods Results Discussion References

1.	Working group on specialist medical training. Hospital doctors: training for the future. London: Department of Health, 1993.
2.	Department of Health. A guide to specialist registrar training. London: HMSO, 1995.
3.	Milligan DWA, Hall D. Implications of the Calman report on child health and paediatrics. Arch Dis Child 1996; 74: 260-263[Free Full Text].
4.	Hunter S, McLaren P. Specialist medical training and the Calman report. BMJ 1993; 306: 1281-1282.
5.	Collins M, Ryan R. Single grade specialist training in otolaryngologya survey of attitudes among present and recent trainees. J Laryngol Otol 1994; 108: 291-293[Medline].
6.	Bayley TJ. Specialist training. Med Educ 1995; 29(suppl 1): 95-96.
7.	Bates T. Curricular training and the new deal. Ann R Coll Surg Engl 1996; 78(suppl): 61-62[Medline].
8.	NHS Management Executive. Junior doctors: the new deal. London: Department of Health, 1991.
9.	Charlton B. Service implications of the Calman report. BMJ 1993; 307: 338-339.
10.	Mather HM, Elkeles RS. Attitude of consultant physicians to Calman proposals: a questionnaire survey. BMJ 1995; 311: 1060-1062[Abstract/Free Full Text].
11.	Dempsey GA, Skinner A. Attitudes of consultant anaesthetists to the Calman proposals: a questionnaire survey. Anaesthesia 1997; 52: 181-182[Medline].
12.	Johnson JN. Survey illustrates misconception of the Calman proposals. BMJ 1995; 311: 1298[Free Full Text].
13.	Temple JG. Attitudes of consultant physicians to Calman proposals. BMJ 1995; 311: 1574[Free Full Text].
14.	Paice E, Pietroni M, Tinker J. Future training of hospital doctors. Br J Hosp Med 1993; 50: 199-201[Medline].
15.	Paice E, West G. Talking to the lost tribes: senior house officer training in North-East Thames. Br J Hosp Med 1994; 51: 123-127[Medline].
16.	General Medical Council. The new doctor. London: GMC, 1997.
17.	Barber P. The colleges, Calman and the new deal. Lancet 1997; 350: 974[CrossRef][Medline].
18.	Hobbs KEF. Specialist training in the UK. Lancet 1997; 350: 1851[Medline].
19.	Paice E, West G, Cooper R, Orton V, Scotland A. Senior house officer training: is it getting better? BMJ 1997; 314: 719-720[Free Full Text].
20.	Allen I, Herzberg J, Hale R, Paice E. Stress in consultants in North Thames. London: Policy Studies Institute, 1999.
21.	Anderson J. The effect of Calman reforms on recruitment, training and service provision. Arch Dis Child 1999; 80: 485-487[Free Full Text].

(Accepted 11 January 2000)