BMJ  2003;327:139-142 (19 July), doi:10.1136/bmj.327.7407.139

Learning in practice

A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study

¹ Department of Psychology, University College London, London WC1E 6BT, ² London SW20 9DY

Abstract

Objective To assess whether A level grades (achievement) and intelligence (ability) predict doctors' careers.

Design Prospective cohort study with follow up after 20 years by postal questionnaire.

Setting A UK medical school in London.

Participants 511 doctors who had entered Westminster Medical School as clinical students between 1975 and 1982 were followed up in January 2002.

Main outcome measures Time taken to reach different career grades in hospital or general practice, postgraduate qualifications obtained (membership/fellowships, diplomas, higher academic degrees), number of research publications, and measures of stress and burnout related to A level grades and intelligence (result of AH5 intelligence test) at entry to clinical school. General health questionnaire, Maslach burnout inventory, and questionnaire on satisfaction with career at follow up.

Results 47 (9%) doctors were no longer on the Medical Register. They had lower A level grades than those who were still on the register (P < 0.001). A levels also predicted performance in undergraduate training, performance in postregistration house officer posts, and time to achieve membership qualifications (Cox regression, P < 0.001; b=0.376, SE=0.098, exp(b)=1.457). Intelligence did not independently predict dropping off the register, career outcome, or other measures. A levels did not predict diploma or higher academic qualifications, research publications, or stress or burnout. Diplomas, higher academic degrees, and research publications did, however, significantly correlate with personality measures.

Conclusions Results of achievement tests, in this case A level grades, which are particularly used for selection of students in the United Kingdom, have long term predictive validity for undergraduate and postgraduate careers. In contrast, a test of ability or aptitude (AH5) was of little predictive validity for subsequent medical careers.

Introduction

Selection of UK medical students depends mainly on grades achieved in school leaving examinations, such as A levels.¹ Few long term studies have validated such selection measures,² and their theoretical underpinning is unclear. Examinations measure achievement, accomplishment, or attainment and assess whether students have mastered an academic subject. In contrast, measures of ability or aptitude assess cognitive ability independently of cultural content and educational experience and are typified by measures of intelligence (general mental ability³) (see www.bmj.com). Whereas intelligence shows stability through life,⁴ achievement tests depend mainly on recent educational experience.

Although seldom articulated, three arguments underpin selection with achievement tests:

• The achievement argument—A levels ensure a minimum competence in the sciences basic to medicine, such as chemistry and biology
  
• The ability argument—Academic success depends mainly on intellectual ability,² and achievement tests indirectly assess intelligence. Because achievement tests can be biased or inaccurate, due to poor schooling, absent role models, low expectations, or inappropriate motivation, there is a case for replacing A levels with measures of aptitude or ability^{5 6}
  
• The motivation argument—A levels are effective because university education requires not only intellectual ability but also good study skills and motivation. High A level grades indicate both satisfactory intellectual ability and learning style. The content of the course therefore matters less than the fact of success.
  

To distinguish such positions we need to relate career outcomes to achievement and intellectual ability. In 2002 we followed up a cohort of clinical students who had taken a standard intelligence test when they entered Westminster Medical School between 1975 and 1982. We had four outcome measures:

• Dropout—Whatever the problems of defining success in a medical career, doctors not on the Medical Register are not successful as practising doctors, albeit that non-clinicians provide much benefit to medicine and society
  
• Career progression—Medical careers are hierarchical. Speed of progression and of attaining postgraduate qualifications therefore indicate success. Although exceptions occur, doctors who take longer to reach the top realise their potential less
  
• Research output—Many doctors publish research and some publish a lot. The implicit presumption is that productive research is the prerogative of the brightest and the best (and typically is the basis for MB-PhD selection)
  
• Stress, burnout, and satisfaction with medicine as a career—A successful doctor is a happy doctor, with low stress and burnout and high career satisfaction. Although less intellectually able doctors may suffer stress due to difficulties in keeping up to date as practice changes, a more subtle converse argument suggests that stressed doctors are those with highest ability, day to day practice providing insufficient variation for adequate intellectual stimulation.⁷ patient interaction.
  

For practical reasons we could not assess doctor-patient interaction.

Method

PRF administered the AH5 (a timed "high grade" intelligence test⁸) to clinical students entering the Westminster Medical School from 1975 to 1982. The test has measures of verbal and reasoning ability (part I, "verbal") and spatial ability (part II, "spatial"). Students were informed that the test was confidential and for research and that results would not be available to teachers or examiners.

In 1988, PRF and ICM collated the results with date of birth, sex, A levels, intercalated degree results, finals performance, and performance in preregistration posts.⁹ A levels were scored as A=5 to E=1 and O/F=0, and summarised as the mean. Performance at finals was recorded as 4=distinction, 3=pass all first time, 2=pass after resits, 1=fail. Preregistration performance was the average consultant rating (4=outstanding, 3=good, 2=satisfactory, 1=inadequate).

In 2001 we used the Medical Register and Directory to trace the graduates. In January 2002 we sent a questionnaire to those on the 2001 UK Medical Register; non-respondents were sent two reminders. The questionnaire asked about career, qualifications, interests, and personality.^{10 11} We assessed stress with the general health questionnaire (GHQ-12) and an abbreviated Maslach burnout inventory (aMBI),¹² with additional questions on satisfaction with medicine (see www.bmj.com). Statistical analysis used SPSS 10.0 and LISREL 8.51.

Results

The mean total AH5 score (fig A, bmj.com) of 40.4 was similar to norms,⁸ as were verbal and spatial scores (table A, bmj.com). The mean A level score (fig A, bmj.com) was 4.00, equivalent to grade BBB. AH5 score and A level grade were correlated (Pearson r=0.285, P < 0.001; fig B, bmj.com).

Dropouts from Medical Register—All 511 students registered with the General Medical Council, but only 464 were on the 2001 Medical Register. The 47 doctors who left the register (a mean of 11.1 years after qualifying; SD 5.9; range 2-23) had lower A level grades but not lower AH5 scores (table A, bmj.com); see www.bmj.com for ROC analysis. Two doctors subsequently returned to the register. Of the remainder, three had died, contact details were available for 35, and no information was available for seven.

Questionnaire response—Of the 464 doctors on the register, 349 (73%) replied to the questionnaire. Non-respondents had lower AH5 scores but did not have different A levels results (table A, bmj.com).

Career choice and career progression—Of 332 doctors for whom we had usable information, 173 worked in hospital (149 were consultants) and 131 in general practice (116 were principals). Of the remainder, four were not working, five had non-medical posts, and 19 had other medical posts. Hospital doctors had higher A level grades and AH5 scores (see table A on bmj.com), each effect being significant after we accounted for the other (A levels: Student's t test, t₂₉₉ = 2.674, P=0.008; AH5: t₂₉₉ = 2.059, P=0.040). Remaining analyses therefore took differences in speciality into account. Figure 1 shows the career progression of hospital doctors and general practitioners. Qualifications are grouped into memberships (MRCP, FRCS, etc), diplomas (or equivalent, often offered by Royal Colleges), and academic degrees (PhD, MD, masters, or bachelors degree). A levels had a highly significant effect on years to membership (table, Cox regression, P < 0.001; fig 2), even after we accounted for AH5 (P=0.001). AH5 had a significant simple effect on years to membership (P=0.049) but not after we accounted for A levels (P=0.401). Other effects of A levels and AH5 were not significant after we accounted for multiple testing.

View larger version (39K): [in this window] [in a new window]   Fig 1 Careers of doctors in hospital medicine and general practice (career grade above and acquisition of memberships, diplomas, and academic degrees below)

 

View larger version (29K): [in this window] [in a new window]   Fig 2 Kaplan-Meier plots for percentage of doctors obtaining membership in relation to A level grade, after taking hospital/general practice differences into account

 

Structural modelling of educational achievement—We modelled academic and professional achievement using structural equation modelling with causal order mainly determined by temporal order, except that we regarded AH5 score before A levels. Goodness of fit was excellent (²=4.90, df=8, P=0.768; GFI (goodness of fit index)=0.995; AGFI (adjusted goodness of fit index)=0.988). Each stage predicted the subsequent stage, and A level grade and finals performance had additional direct effects on time to membership (fig 3).

View larger version (13K): [in this window] [in a new window]   Fig 3 Path model of causal associations between different educational achievements of doctors. Coefficients represent standardised path coefficients ( coefficients) with their associated significance levels

 

Research publications—In total 138 doctors (40%) had not published any research papers, 44 (13%) had published 1-2 papers, 36 (11%) 3-5 papers, 30 (9%) 6-10 papers, 39 (11%) 11-20 papers, 18 (5%) 51-100 papers, 29 (9%) 21-50 papers, 8 (2%) had published more than 100 papers. Regression of normal scores (ranked normal deviates; normal order statistics) showed differences and between hospital doctors and general practitioners (P < 0.001) but no effect of A levels or AH5 score (table).

View this table: [in this window] [in a new window]   Effects of mean A level grade, total AH5 score, and AH5 verbal and spatial subscores on various outcome measures. All effects are simple effects that do not take other variables into account; all analyses, however, take differences in general practice/hospital into account. Analyses for time to event use Cox regression and other analyses use multiple regression. Figures are regression b (SE) and P values

 

Stress, burnout, and satisfaction with medicine as a career—Sixty two doctors (18%) scored 4 on the general health questionnaire, indicating "caseness" for stress. General practitioners scored higher than hospital doctors on measures of emotional exhaustion, depersonalisation, and personal accomplishment in the Maslach burnout inventory but did not differ on the general health questionnaire (0-1-2-3 scoring) or on satisfaction with a medical career. No measure showed any association with A level grades or AH5 score (table).

Discussion

Few studies have attempted to validate the selection procedures for medical students, although in such studies the effect size of academic measures for postgraduate performance is 0.48.² Despite A levels being the basis for selection in the United Kingdom,¹ little evaluation has taken place, and although occasional comments suggest that A levels are "completely unpredictive" they actually predict early dropout from medical school.^{13 14} For university degrees overall, A levels also predict degree class, dropout, and repeated years, particularly for science.^{13 14} We have shown that A level results, which are measures of achievement, can predict time taken to gain membership qualifications, choosing to become a general practitioner, and leaving the register. In contrast the AH5, which measures ability, cannot independently predict membership qualifications or dropout.

A levels therefore have validity in selection, with a validity coefficient of about 0.3 (see www.bmj.com), although care should be taken in generalising the results to other examinations in other countries. Intelligence does not predict careers, thus rejecting the ability argument. A levels predict because they assess achievement, and the structural model shows how past achievements predict future achievement. Our data cannot distinguish the achievement argument and the motivation argument, although the long term, direct effect of A levels on membership examinations (fig 3) suggests that motivation might be important.

Despite their predictive ability, A levels are probably not the only predictors² and should not be the sole basis for selection.¹⁵ Some of our other outcomes were not predicted by A levels but were correlated with measures of personality (see www.bmj.com) and would probably also be predicted by learning styles.^{16 17} West answered Smith's editorial question of "Why are doctors so unhappy?"¹⁸ by suggesting that burn out because they are overqualified for a repetitious job.⁷ The causes of stress and burnout in doctors are complex,¹² but our data suggest that excess intellectual ability is not one of them.

What is already known on this topic There are few prospective studies of achievement tests used in student selection, such as A levels, in relation to outcomes in medical careers It is not clear whether A levels are useful in selection because they assess knowledge, motivation, study habits, or ability (intelligence) What this study adds A level results can predict outcome in medical careers An ability test (the AH5 intelligence test) does not predict outcome It is not clear yet whether the predictive value of A levels results from assessing knowledge, motivation, or study habits Other measures such as personality are also probably important in predicting outcome doctors

---

Dr Fleming died on 1 December 2002

Supplementary definitions, more detailed results, three extra tables, and three extra figures are on bmj.com

We thank Robin A M Forrest, secretary of Westminster Medical School, for his help in collecting the original data; Naomi Tunnicliffe for her assistance with this project, Eamonn Ferguson for helpful comments on a draft of the manuscript, and Rod Rhyss-Jones and Urmila Weller of Imperial College School of Medicine for their help in tracing doctors. We especially thank the Westminster Medical School graduates who helped with this study, which was an intercalated BSc project carried out by ES and PP.

Contributors: PRF initiated the study, and was responsible for collecting the original data from 1975 to 1982, and ICMcM and PRF collated those data in 1989. ICMcM, ES, and PP designed the present follow up, and ES and PP traced and contacted the doctors. AK was responsible for data coding and entry. ICMcM, ES, PP, and AK jointly carried out data analysis. The first draft of the paper was written by ICMcM, and ES, PP, AK, and PRF contributed to its revision. ICMcM is guarantor.

Funding: AK was supported by funding from North Thames Postgraduate Medical and Dental Education.

Competing interests: None declared.

References

1. McManus IC. Factors affecting likelihood of applicants being offered a place in medical schools in the United Kingdom in 1996 and 1997: retrospective study. BMJ 1998;317: 1111-6.[Abstract/Free Full Text]
2. Ferguson E, James D, Madeley L. Factors associated with success in medical school and in a medical career: systematic review of the literature. BMJ 2002;324: 952-7.[Free Full Text]
3. Schmidt FL, Hunter JE. The validity and utility of selection methods in personnel psychology: practical and theoretical implications of 85 years of research findings. Psychol Bull 1998;124: 262-74.[CrossRef][Web of Science]
4. Deary IJ. Differences in mental abilities. BMJ 1998;317: 1701-3.[Free Full Text]
5. McDonald AS, Newton PE, Whitton C, Higgs SE. A pilot of aptitude testing for university entrance. London: National Foundation for Educational Research, 2000.
6. McDonald AS, Newton PE, Whetton C, Benefield P. Aptitude testing for university entrance: a literature review. London: National Foundation for Educational Research, 2000.
7. West PA. Calibre of people recruited to medicine may be too high for the job [letter]. BMJ 2001;322: 1361.[Free Full Text]
8. Heim AW. AH5 group test of high-grade intelligence. Windsor: National Foundation for Educational research (NFER), 1968.
9. McManus IC, Tunnicliffe N, Fleming PR. The independent effects of intelligence and educational achievements in predicting final examination success (abstract). Med Educ 1990;24: 181-4.[Web of Science][Medline]
10. Costa PT, McCraw RR. Revised NEO personality inventory and NEO five-factor inventory: professional manual. Odessa, FL: Psychological Assessment Resources, 1992.
11. Matthews G, Deary IJ. Personality traits. Cambridge: Cambridge University Press, 1998.
12. McManus IC, Winder BC, Gordon D. The causal links between stress and burnout in a longitudinal study of UK doctors. Lancet 2002;359: 2089-90.[CrossRef][Medline]
13. Peers IS, Johnston M. Influence of learning context on the relationship between A-level attainment and final degree performance: a meta-analytic review. Br J Educ Psychol 1994;64: 1-17.
14. Bekhradnia B, Thompson J. Who does best at university? London: Higher Education Funding Council England, 2002. www.hefce.ac.uk/Learning/whodoes
15. Universities UK. Fair enough? Wider access to university by identifying potential to succeed. London: Universities UK, 2003.
16. McManus IC, Richards P, Winder BC, Sproston KA. Clinical experience, performance in final examinations, and learning style in medical students: prospective study. BMJ 1998;316: 345-50.[Abstract/Free Full Text]
17. McManus IC, Richards P, Winder BC. Intercalated degrees, learning styles, and career preferences: prospective longitudinal study of UK medical students. BMJ 1999;319: 542-6.[Abstract/Free Full Text]
18. Smith R. Why are doctors so unhappy? BMJ 2001;322: 1073-4.[Free Full Text]

(Accepted May 6, 2003)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Articles

Widening participation in medicine

Pamela B Garlick and Gavin Brown
BMJ 2008 336: 1111-1113. [Extract] [Full Text] [PDF]

Should all medical students be graduates first? Yes

Ed Peile
BMJ 2007 335: 1072. [Extract] [Full Text] [PDF]

Should all medical students be graduates first? NO

Charles George
BMJ 2007 335: 1073. [Extract] [Full Text]

Predicting the "strugglers": a case-control study of students at Nottingham University Medical School

Janet Yates and David James
BMJ 2006 332: 1009-1013. [Abstract] [Full Text] [PDF]

Admissions processes for five year medical courses at English schools: review

Jayne Parry, Jonathan Mathers, Andrew Stevens, Amanda Parsons, Richard Lilford, Peter Spurgeon, and Hywel Thomas
BMJ 2006 332: 1005-1009. [Abstract] [Full Text] [PDF]

Intellectual aptitude tests and A levels for selecting UK school leaver entrants for medical school

I C McManus, David A Powis, Richard Wakeford, Eamonn Ferguson, David James, and Peter Richards
BMJ 2005 331: 555-559. [Full Text] [PDF]

School exam results matter in medical job applications

R E Weir, F H Zaidi, D E J Whitehead, and R E MacLaren
BMJ 2004 328: 585. [Extract] [Full Text]

Do school exams predict doctors' success?: Career achievements are not only measure

Nick Myerson
BMJ 2003 327: 810. [Extract] [Full Text]

Do school exams predict doctors' success?: Is it not a retrospective study?

Rohit Chaturvedi and Shailesh Chaturvedi
BMJ 2003 327: 810. [Extract] [Full Text]

Intelligence

Upasana Tayal
BMJ 2003 327: 170. [Extract] [Full Text]

School exam results predict success in medicine
BMJ 2003 327: 0. [Full Text]

This article has been cited by other articles:

• Hudson, N. P. H., Rhind, S. M., Moore, L. J., Dawson, S., Kilyon, M., Braithwaite, K., Wason, J., Mellanby, R. J. (2009). Admissions processes at the seven United Kingdom veterinary schools: a review. Vet Rec. 164: 583-587 [Abstract] [Full text]  
• Garlick, P. B, Brown, G. (2008). Widening participation in medicine. BMJ 336: 1111-1113 [Full text]  
• Peile, E. (2007). Should all medical students be graduates first? Yes. BMJ 335: 1072-1072 [Full text]  
• George, C. (2007). Should all medical students be graduates first? NO. BMJ 335: 1073-1073 [Full text]  
• Breit, S. M, Kunzel, W. (2007). Effect of the Recently Established Admissions Procedure on Success in the First-Year Exams at the University of Veterinary Medicine Vienna, Austria. jvme 34: 335-339 [Abstract] [Full text]  
• Kunzel, W., Breit, S. M. (2007). Admissions Procedures at the University of Veterinary Medicine Vienna, Austria. jvme 34: 639-644 [Abstract] [Full text]  
• Parry, J., Mathers, J., Stevens, A., Parsons, A., Lilford, R., Spurgeon, P., Thomas, H. (2006). Admissions processes for five year medical courses at English schools: review. BMJ 332: 1005-1009 [Abstract] [Full text]  
• Yates, J., James, D. (2006). Predicting the "strugglers": a case-control study of students at Nottingham University Medical School. BMJ 332: 1009-1013 [Abstract] [Full text]  
• McManus, I C, Powis, D. A, Wakeford, R., Ferguson, E., James, D., Richards, P. (2005). Intellectual aptitude tests and A levels for selecting UK school leaver entrants for medical school. BMJ 331: 555-559 [Full text]  
• Peile, E., Carter, Y. (2005). Selecting and supporting contented doctors. BMJ 330: 269-270 [Full text]  
• Weir, R E, Zaidi, F H, Whitehead, D E J, MacLaren, R E (2004). School exam results matter in medical job applications. BMJ 328: 585-585 [Full text]  
• Myerson, N. (2003). Do school exams predict doctors' success?: Career achievements are not only measure. BMJ 327: 810-810 [Full text]  
• Chaturvedi, R., Chaturvedi, S. (2003). Do school exams predict doctors' success?: Is it not a retrospective study?. BMJ 327: 810-810 [Full text]  

Rapid Responses:

Read all Rapid Responses

Social intelligence and patient satisfaction need to be taken into account

Dagmar M Haller-Hester
bmj.com, 18 Jul 2003 [Full text]

A Levels and Medical Careers

Bleddyn Jones
bmj.com, 18 Jul 2003 [Full text]

Career achievement + stablility outwith work = True success=

Egbert Pravinkumar
bmj.com, 18 Jul 2003 [Full text]

Dropout of uncertain significance?

Donald S Mackie
bmj.com, 19 Jul 2003 [Full text]

The more things change, the more they stay the same

Kenneth Wong
bmj.com, 20 Jul 2003 [Full text]

Re: Dropout of uncertain significance?

Peter A Wynn Owen
bmj.com, 21 Jul 2003 [Full text]

Career success and vocational success

Nick Myerson
bmj.com, 21 Jul 2003 [Full text]

Gender may have been a predictor too.

Christine Bundy
bmj.com, 22 Jul 2003 [Full text]

Bias in the study

seshubabu gosala
bmj.com, 22 Jul 2003 [Full text]

Circumstances predict careers!

sandya p. iyer
bmj.com, 24 Jul 2003 [Full text]

Is it not a retrospective study?

Rohit Chaturvedi, et al.
bmj.com, 26 Jul 2003 [Full text]

Surprise, surprise

Bruno J Rushforth
bmj.com, 29 Jul 2003 [Full text]

What this study does not add

Guido Grass, et al.
bmj.com, 30 Jul 2003 [Full text]

A levels and research potential

Donald R. Forsdyke
bmj.com, 12 Aug 2003 [Full text]

Good A Levels don't always produce good doctors

john robert martin
bmj.com, 13 Aug 2003 [Full text]

A level grades and undergraduate academic performance in Hong Kong

Kent-Man Chu, et al.
bmj.com, 15 Aug 2003 [Full text]

Neuroticism May Predict Personal Success

Michele G Daly, et al.
bmj.com, 15 Sep 2003 [Full text]

Predictive Validity of the SSC, HSC and Medical College Admission Test for Academic Performance

Lubna A Baig
bmj.com, 23 Sep 2003 [Full text]

Online poll

Find out more on doc2doc >>