Intended for healthcare professionals

Papers

Factors affecting likelihood of applicants being offered a place in medical schools in the United Kingdom in 1996 and 1997: retrospective studyCommentary: League tables will helpCommentary: Some legal aspects arising from the study

BMJ 1998; 317 doi: https://doi.org/10.1136/bmj.317.7166.1111 (Published 24 October 1998) Cite this as: BMJ 1998;317:1111

This article has a correction. Please see:

  1. I C McManus, professor of psychology and medical education. (i.mcmanus{at}ucl.ac.uk)
  1. Centre for Health Informatics and Multi-professional Education, Royal Free and University College Medical School, University College London, Whittington Hospital Campus, London N19 5NF
  2. Medical Practitioners' Union, London SE1 1UN
  3. Gray's Inn, London WC1R 5AY
  • Accepted 5 October 1998

Abstract

Objective To assess the relation between a range of measures and the likelihood of applicants to medical schools in the United Kingdom being offered a place overall and at each medical school, with particular emphasis on ethnic minority applicants.

Design Data provided by the Universities and Colleges Admissions Service on 92 676 applications to medical schools from 18 943 candidates for admission in 1996 and 1997. Statistical analysis was by multiple logistic regression.

Main outcome measures Receipt of a conditional or unconditional offer of a place at medical school.

Results Eighteen separate measures were independently associated with the overall likelihood of receiving an offer. Applicants from ethnic minority groups were disadvantaged, as were male applicants, applicants applying late in the selection season, applicants making non-medical (so called insurance) choices, applicants requesting deferred entry (so called gap year), and applicants at further or higher education or sixth form colleges. Analysis at individual medical schools showed different patterns of measures that predicted offers. Not all schools disadvantaged applicants from ethnic minority groups and the effect was stable across the two years, suggesting structural differences in the process of selection. The degree of disadvantage did not relate to the proportion of applicants from ethnic minority groups.

Conclusions The data released by the Council of Heads of Medical Schools allow a detailed analysis of the selection process at individual medical schools. The results suggest several areas in which some candidates are disadvantaged, in particular those from ethnic minority groups. Similar data in the future will allow monitoring of changes in selection processes.

Editorial by Abbasi and Education and debate p   11149

Key messages

  • The Council of Heads of Medical Schools has made publicly available, on the website of the Universities and Colleges Admissions Service, detailed data on individual applications for medical school in 1996 and 1997

  • >These data allow analysis of factors influencing selection at individual medical schools in the United Kingdom, although some important measures such as GCSE grades, estimated A level grades, and assessments of personal attributes are not available

  • In 1996-7 certain groups, in particular ethnic minority groups and male applicants, were disadvantaged during selection

  • The disadvantage of applicants from ethnic minority groups seems stable across years, with some schools consistently showing no evidence of disadvantage

  • Provision of similar data in the future will allow continued monitoring of selection and of the proposals for change made by the council

Introduction

The importance of annually monitoring the admissions procedure within each medical school cannot be over-emphasised. It is naturally desirable that factors such as medical parents, social class, private education, race, and sex do not become significant predictors of shortlisting or interview success, whether unconsciously determined or otherwise.1

The Council of Heads of Medical Schools has recently decided to place anonymised data for individual applications to medical schools in the United Kingdom on the website of the Universities and Colleges Admissions Service (www.ucas.ac.uk).For the first time much of the information that is important in selecting medical students can be examined in detail. The data set is large and complex (4.5 million pieces of information on 93 000 applications from 19 000 candidates over two years).In this paper I summarise measures thatare related to receiving an offer at individual medical schools and overall. These measures have been important in previous studies25or are of intrinsic interest. Not all potentially important measures were available. For example, computer readable data were not available on grades at GCSE (general certificate of secondary education, the examinations generally taken at the age of 16); predicted grades at A level (the examinations generally taken at the age of 18); the textual statements provided by applicants and referees; or the selection process within medical schools, including shortlisting and interviewing, which are often used to assess motivation and personality. A necessarily brief paper cannot do complete justice to such rich data, and a more detailed report is available on the website of the Universities and Colleges Admissions Service(http://www.ucas.ac.uk/)computer programs for carrying out additional statistical analyses are available on request from me.

The two aims of this paper are to provide an overview of measures important in selection and tolook in detail at the controversial issue of the selection of applicants from ethnic minority groups.69 Important methodological concerns, to avoid the flaws of some previous studies, 1011 are adequate control of confounding variables and analysis of offers rather than entrants (since candidates themselves select between schools when holding several offers12

Methods

The Universities and Colleges Admissions Service provided anonymised data for all applications to medical school in the 1996 and 1997 cycles—that is, applications during autumn 1995 and 1996—for applicants with permanent residence in the United Kingdom (home applicants). During 1996 and 1997 applicants could make a maximum of six applications (although they were recommended to make five). Deferred entry (or the so called gap year) was known only for the 1997 data. The service sent non-anonymised data to medical schools for checking. Ethnic origin was self classified by applicants using standard census categories.

Table 1

Effects of measures in overall analysis of data on candidates in 1996 and 1997 combined

View this table:

Data were supplied as an EXCEL 5.0 workbook, which was analysed using SPSS for Windows 6.1 (syntax and system files are available on request from me). Separate SPSS files were produced for analyses of candidates and applications.

Statistical analysis used multiple logistic regression with simultaneous adjustment of each variable for all other factors. The dependent variable for the analysis of candidates was receipt of one or more offers and for the analysis of applications receipt of an offer. Missing values were handled by imputation of population means.13

A levels and Scottish highers— Most applicants to medical schools outside Scotland take A levels after application, whereas about half of applicants to Scottish medical schools take highers, which are taken before application. Selectors typically have no information on achieved grades at A level but only GCSE grades and predicted grades at A level, which were not available in this study. In this study, however, academic achievement is described in terms of achieved A level grades, the current standard of sixth form educational achievement. As in previous studies, 2 4 5academic achievement is summarised as the number of A levels and highers taken and the mean grade attained in A levels and highers. Significance testing— Assessing statistical significance is potentially problematic when about 21 factors are assessed at 27 individual schools in two separate years, giving some 1100 tests. A straightforward solution for avoiding α inflation (type I error) treats as significant only results nominally significant in both years, defined as a nominal significance level of <0.01, or <0.05 on one occasion and a geometric mean less than 0.01—for example, P=0.05 and P=0.002. My unpublished report describes many additional analyses, which cannot be described fully in this short paper. When claims seem not to be fully supported by evidence further information is available in the report (available at http://www.ucas.ac.uk/).

Results

Selection overall— Table (1). summarises for the 18 943 candidates in the combined 1996-7 data the relation between the 24 variables and receipt of one or more offers. Eighteen predictors were significant with P<0.05 and 17 with P<0.001. Candidates resitting A levels and mature applicants— Mature applicants and those resitting their A levels are often treated differently during selection. Analysis of the 14 773 who were not mature students or resitting their A levels found only one change in significance level α=0.05): candidates taking an A level in non-science subjects were less likely to receive an offer in the restricted analysis (P=0.040, odds ratio 0.89 (95% confidence interval 0.80 to 0.99)). Applicants taking Scottish highers— Applicants taking only highers might not be properly reflected in the overall analysis. Analysis of offers in the 1225 applicants of known ethnic origin (12.7% from ethnic minority groups; 156/1225) applying to Scottish medical schools found number and grade of highers were both significant predictors (P<0.0001), as were early application (P=0.0013), not putting four or less applications for medicine (P=0.0132), being white (P=0.0232), and not applying from a sixth form or other college (P=0.0004). Ethnic origin— Figure 1 confirms that non-white applicants are less likely to receive an offer across the whole range of A level achievement, with a qualitatively similar effect in applicants taking highers.

Fig 1
Fig 1

Percentage of applicants receiving an offer in relation to educational achievement and ethnic group. Bars show 95% confidence intervals; they are not shown for mean grade in Scottish highers because sample sizes were comparatively small in some groups. Overall difference between white and non-white applicants for highers was significant (P=0.0232) after taking all other factors into account (see text)

Differences between ethnic groups— Figure 2 shows the adjusted odds of disadvantage for each ethnic group. White applicants were morelikely to receive an offer than applicants from any ethnic minority group, with only small differences between ethnic groups. Sex and ethnic origin— The interaction between sex and ethnic origin was significant (P=0.049), women who were not white being less likely to receive an offer than expected. Selection at individual medical schools— Table(2).summarises the significance of each predictor at each medical school. Figure 3 shows the disadvantage of being from an ethnic minority group at each medical school, along with confidence intervals. As with any such league table, results must be interpreted with caution. 14 15The weighted correlation of effects in 1996 and 1997 was 0.428. Disadvantage in 1991 5also correlated (r=0.419) with a composite of disadvantage in 1996-7. Medical schools differ in their proportion of applicants from ethnic minority groups, although figure 4 shows no relation to the extent of disadvantage (r=0.070).

Discussion

Limitations of the study

I have summarised a great deal of information in this paper and have inevitably oversimplified some issues because of a tight timetable and constraints of space. However, this is the first timethe influence of many interrelated factors has been described properly. Placing the data in the public domain means other researchers can explore the data in far more detail. The limitations of the study must be emphasised, particularly that it provides a historical description of selection in 1996 and 1997, not a prediction of future selection, which will certainly change, as will patterns of application. A concern must be with the factors not measured, although, as the nineteenth century neurologist Hughlings Jackson said: “Absence of evidence is not evidence of absence.” Results at GCSE and estimated A level grades might have clarified the interpretation, but their interrelations in a previous study 5suggest that they are unlikely fully to explain differences between white and non-white applicants. Assessments of personal attributes such as motivation, personality, and communicative ability would also have been helpful. Differences in personal attributes between groups might explain the disadvantages found in this study, but such judgments would need to be shown to be reliable, free of bias, and predictive of eventual professional behaviour.

Selection overall

As in the studies of 1981, 1986, and 1991, 2 4 5 many variables predict a candidate receiving offers. Many also raise serious concern. The disadvantage of non-white and male applicants could be construed as evidence under the Race Relations Act 1976 and the Sex Discrimination Act 1975. That applicants taking non-science A levels might be disadvantaged, or that candidates taking AS levels (a sixth form qualification equivalent to half an A level) receive no advantage in selection are unlikely to broaden the intellectual base of medical students outside the modal three sciences (typically mathematics, chemistry, and biology). Reassuringly, general studies at A level does give an advantage to applicants, but selectors sometimes disparage what is seen as “not a real A level” (despite careful standard setting by examining boards). The advantage of applying early suggests a horse race in which some applicants start running before others, and the disadvantage of applicants making a non-medical (insurance) choice 16 sits uncomfortably with medical schools' policy of “no detriment” 17 and the statement, “without prejudice to [applicants' perceived] commitment to medicine” in the handbook of the Universities and Colleges Admissions Service. The disadvantage of applicants requesting deferred entry (gap year) seems at odds with the importance of “encourag[ing] 18-year olds to take a year off before coming to medical school, so they can reflect on what they really want.” 18 Although the much claimed advantage of applicants from independent schools is (again) not confirmed, the clear disadvantage of applicants from further and higher education and sixth form colleges seems unlikely to broaden access to medical education.

Fig 2
Fig 2

Disadvantage of non-white applicants in each of the ethnic groups relative to white applicants (odds ratio of 1 indicated by vertical line)

Table 2
Fig 3
Fig 3

Disadvantage at medical schools in the United Kingdom of non-white applicants in 1996 and 1997 ordered by size of disadvantage in 1997. Horizontal line indicates odds ratio of 1 (no disadvantage). The estimate of disadvantage is corrected for all other variables in the analysis. Oblique dashed line in 1996 indicates disadvantage at each school in 1997. HMS=Hospital Medical School, London; CXWMS=Charing Cross and Westminster Medical School, London; QMW=Queen Mary and Westfield College, London; UMDS=United Medical and Dental Schools of Guy's and St Thomas's Hospitals, London

Fig 4
Fig 4

Disadvantage of applications from ethnic minority groups expressed as odds ratio (y axis) in relation to proportion of applicants to a medical school who were from an ethnic minority group in 1996 and 1997

Selection at individual schools

Inevitably, table (2).will be controversial. However, it reports only results that are significant on a strict criterion and statistical power differs in relation to the number of applicants and offers and the mix of applicants. Also, as figure 3 emphasises, effects are distributed on a continuum, which should not be naively divided into the good, the bad, and the ugly. However, the fact that schools show different effects on each measure is extremely important as it implies that intrinsic, unmeasured differences between groups of applicants are unlikely to explain the overall effects in table 2 and that the differences in table 2 therefore instead reflect structural differences in the processes of selection itself. The long term stability of the disadvantage of ethnic minority groups, with schools such as Bristol, Newcastle, and the Royal Free showing no evidence of disadvantage in 1991, and 1996-7, emphasises that good practice can occur. Correlation of the detailed mechanics of selection with the extent of disadvantage is therefore urgently needed. Space precludes detailed discussion of all differences in table 2, but questions about the ethics of selection can be raised through the touchstone of the advantage of local applicants. Such an advantage may be seen as justifiable if it reinforces a strong link between medical school and local community in the provision of medical services. It is less desirable, however, if it inadvertently disadvantages a non-local applicant who might not be adequately informed of such a policy—for example, in a clearly stated code of practice.19

Racism, discrimination, and disadvantage

These data may provoke claims that medical schools are racist (and perhaps also sexist). The Commission for Racial Equality defines racism in terms of attitudes, and other definitions consider it as ideology20; in so far as this study does not assess the attitudes or ideologies of selectors, little further can be said. In terms of behavioural outcome, there is no doubt that applicants from ethnic minority groups are disadvantaged. Whether that disadvantage is due to discrimination cannot be shown from these data alone. However, in conjunction with previous studies and other literature, it seems reasonable to conclude that in some cases at some schools discrimination is occurring, wittingly or unwittingly. Direct discrimination seems unlikely (but not impossible21, although an aberrant minority of selectors may be subverting an otherwise fair system. Indirect discrimination is more possible, perhaps either through assessment of A level achievement indirectly from estimated grades and GCSE results or from assessing motivational and personality factors indirectly through achievements and experiences with different meanings in different cultural groups.

The important and correct decision by the Council of Heads of Medical Schools to release detailed information on the process of medical student selection will inevitably result in a debate about how selection should take place. This study may be the start of that process, but it is certainly not the end; other researchers will dig deeper into these data and will analyse other evidence of their own. My findings may be uncomfortable reading for some medical schools and some medical school applicants. Nevertheless, putting the information in the public domain was undoubtedly the correct decision since open scrutiny is the only way that democratic societies can reassure themselves that those acting on their behalf are doing so appropriately. Proper information is therefore the first step towards improving selection. This analysis also provides a precedent for selection into other courses, professions, and occupations, none of which has yet provided data of equivalent richness and detail. The Council of Heads of Medical Schools is committed to change. The release of future data on selection will allow monitoring of those changes.

Acknowledgments

I thank the Council of Heads of Medical Schools, its chairman and executive secretary, and the deans of individual medical schools for allowing me access to these data before full publication. I also thank Mr Richard Coleman and Ms Liz Viggars of the Universities and Colleges Admissions Service and Mr Michael Powell of the Council of Heads of Medical Schools for their help in analysing thedata and preparing the results. Dr Sheila Gore reviewed a previous statistical analysis and gave several helpful suggestions. Deans and admissions tutors at several medical schools also gave useful comments on an earlier draft of this paper and I thank them for their care and attention to detail, which has undoubtedly clarified several aspects of this paper.

Contributor: ICM was commissioned by the Council of Heads of Medical Schools as an experienced independent academic researcher to analyse the 1996 and 1997 selection data before public release. The data were provided by the Universities and Colleges Admissions Service. The analysis and drafting of paper and reports was carried out entirely by ICM. Earlier drafts of the report and this paper were modified in response to comments made by admissions tutors and deans, and some members of the Council of Heads of Medical Schools have seen and commented on a near final draft of this paper. The views expressed here are the professional opinions of ICM and are not necessarily the opinions of the Council of Heads of Medical Schools.

Funding: The Council of Heads of Medical Schools commissioned this study.

Competing interests: None declared.

References

Commentary: League tables will help

  1. Aneez Esmail, vice president. (aneez.esmail{at}man.ac.uk)
  1. Centre for Health Informatics and Multi-professional Education, Royal Free and University College Medical School, University College London, Whittington Hospital Campus, London N19 5NF
  2. Medical Practitioners' Union, London SE1 1UN
  3. Gray's Inn, London WC1R 5AY

    The decision by the Council of Heads of Medical Schools to release data on the outcome of the selection process at medical schools in the United Kingdom is to be welcomed. An important question, however, is why have the data, which have been available since 1990, only just been released?

    The key to dealing with the discrimination that McManus has documented is for medical schools to publish and release into the public domain their selection criteria and to monitor the impact ofthose criteria on issues such as sex and ethnic group on a yearly basis. There is nothing like a league table to concentrate the mind, especially if your institution is a persistent offender in discriminating against students from ethnic minority groups. Openness should be the overriding principle.

    As with any analyses, there are imperfections. However, the time for excuses is over because onthe basis of the evidence presented by McManus there should be prima facie evidence of a case for investigation by the Commission for Racial Equality. Unfortunately, discrimination in medical school applications is not the only part of the NHS where there is a problem. Discrimination against candidates from ethnic minority groups in job applications at all levels in the NHS has already been documented. 1 2 There is also a problem of discrimination in the complaints brought before the General Medical Council 3 and in the allocation of merit awards. 4 Observers may legitimately argue that racism is almost institutionalised in the NHS.

    Some may argue that discrimination is not a problem since students from ethnic minority groups,especially Asians, are overrepresented (compared with the general population) among medical school students. The issue is, however, one of equality of opportunity. If selection criteria are set and candidates meet those criteria then why does it matter if a large number of students from ethnic minority groups are selected? The students affected are British citizens, born in the United Kingdom, speaking English in all its regional dialects. When they qualify most of their patients will be white and it makes no difference in terms of the clinical care that they deliver what the colour of their skin is. We cannot go down the road of quotas and attempts to create proportionality in the medical workforce as is happening in the United States, where in some states Asian students are set higher standards for entry because of the perception that there are “too many of them.” 5

    There is evidence of good practice in the selection of medical students. Institutions that are currently in the top half of the league table of discrimination against ethnic minority groups listed by McManus need to learn from what is happening at universities such as Newcastle and Birmingham, where there seems to be no such problem.

    If the problem of racism is structural then structures need to be broken down. Sometimes radical solutions are required, and it is better that these are developed internally rather than imposed by the courts. The end result has to be selection based on the broadest definition of merit compatible with producing good competent doctors. It has little to do with the colour of one's skin.

    References

    Commentary: Some legal aspects arising from the study

    1. Marie Demetriou, barrister.
    1. Centre for Health Informatics and Multi-professional Education, Royal Free and University College Medical School, University College London, Whittington Hospital Campus, London N19 5NF
    2. Medical Practitioners' Union, London SE1 1UN
    3. Gray's Inn, London WC1R 5AY

      In this commentary I will give a legal context to the paper by McManus and highlight the main issues of law that arise from its findings. In doing this, I have focused on two key conclusions of the study—namely, that in 1996-7 both male applicants and applicants from ethnic minority groups were disadvantaged in their applications to medical schools.

      The starting point is that, in deciding whether to admit a candidate, it is unlawful for universities to discriminate on grounds of sex or race. This follows from section 22 of the Sex Discrimination Act 1975 and section 17 of the Race Relations Act 1976. The prohibition on sex discrimination applies to treatment that disadvantages men just as much as it does to treatment that is unfavourable towards women. The principles underlying discrimination and the steps entailed in proving or defending a claim under each act are largely identical.

      Statistical differentials such as those revealed by this study are often used to form the basisof a discrimination claim. They may be used to raise a prima facie case, which it will then be for the relevant institution to defend. Such a claim may take one of two forms: direct or indirect discrimination. The distinction is important because, generally, direct discrimination, in contrast to indirect discrimination, is not capable of justification. Direct discrimination arises where but for a person's sex or race he or she would be treated more favourably. By contrast, indirect discrimination arises not when a person's sex or race is the determinative factor but when a condition or standard is applied that is more difficult for a person of one sex or of a certain race to satisfy.

      These statistics are not likely to be indicative of direct discrimination. They are much more likely to raise issues of indirect discrimination. A person bringing such a claim would have to point to a condition or criterion applied by medical schools that is less easily satisfied by men or by applicants from ethnic minority groups and which thereby leads to men or ethnic minority groups being disadvantaged. The relevant university would then have to show that the condition or criterion at issue was objectively justified. The courts have held that such justification requires an objective balance between the discriminatory effect of the condition and the reasonable needs of the party who applies the condition. For example, an applicant from an ethnic minority group might be able to show that medical schools placed significant weight on A level predictions when deciding whether to offer places. The applicant may then go on to prove that applicants from ethnic minority groups received lower predictions than white applicants. It would then be for the medical school concerned to justify the use of A level predictions as an entry criterion.

      These are the potential legal issues that arise from the conclusions of this study. Although, as I have noted, statistics of this nature can and do form the basis of discrimination claims, the limits on their evidential value must be borne in mind. The weight of this study in indicating discrimination may well be altered by placing it in the context of a study covering a greater number of years.

      View Abstract