Introduction

Top Abstract Introduction Methods Results Discussion References

In the United Kingdom there is huge competition to study medicine, with in excess of 13 000 applications through the Universities and Colleges Admissions Service for just over 4000 places. A series of recent studies has found that the likelihood of success is less among applicants from ethnic minority groups than among white applicants.^1-6 A study based on 1991 entry indicated that the situation was improving.⁵ Even so, when seven other mainly academic aspects of the application were taken into account not being from an ethnic minority group remained a significant predictor of success. A recent study looking at all home applicants for entry in 1996 and 1997 found a greater disadvantage for applicants from ethnic minority groups than previously.⁶ This study was, however, limited for technical reasons by not being able to include data on GCSE grades, which form a large part of the selection process and which were important predictors of success in previous studies.⁵

Most unsuccessful applicants are rejected solely on assessment of their application formthat is, at the `shortlisting' stage before being invited for interview. It is during shortlisting that students from ethnic minority groups are believed to be disadvantaged.⁵ The application form contains no explicit reference to the applicant's ethnic background, so it seems likely that any discrimination must be based on the applicant's name. For this reason it has been suggested that the whole of the shortlisting process be performed anonymously.⁵

We decided to assess the feasibility of assessing forms anonymously within the current admissions system of the Universities and Colleges Admissions Service. In addition, we assessed the impact of doing so on the shortlisting system we have used at Leeds school of medicine for the past four years.

	Methods

Top Abstract Introduction Methods Results Discussion References

Shortlisting process
Our shortlisting process involves each application form being assessed separately by two of three admissions tutors (including AL). A score from zero to 20 points is awarded made up of four components including career insight (4 points), non-academic activities (6 points), academic profile (4 points), and suitability for a medical career as described by the confidential reference (6 points). When assessing applications, admissions tutors are unaware of the other selector's score. The sum of the two scores then forms the sole basis of the decision to reject, accept, or interview the applicant, although the threshold values may vary throughout the year according to the numbers and quality of applications received.

Anonymising forms
We studied all home (including EU) applicants to the medical course for entry in 1998 except for graduate applications, which are assessed separately. An admissions clerk, who played no part in the assessment of applicants, anonymised forms. All text to be deleted was first overwritten with a red marker pen, which allowed the text underneath to be easily read, and this comprised the `open' application. The form was then photocopied, which rendered the highlighted section indecipherable, and this comprised the `blind' form. The following text was deleted: full name, email address, country of birth, applicant's signature, and all references to the applicant's name found in the personal statement or confidential reference sections. Batches of about 100 forms with alternate blind and open applications were sent to each selector, with care being taken to avoid any selector assessing the same applications twice. In this way selectors always read alternate open and blind forms, of different applicants, and each form was assessed both open and blind by two different selectors. The short time available for processing of applications and other commitments by selectors prohibits equity in the number of forms assessed and equal randomisation of pairs of selectors.

Assignment of applicants to ethnic group
Ethnic background was determined by two administrative clerks not concerned with the selection process. From the full name, applicants with non-European sounding names were classified as from an ethnic minority group and only coded as such when both clerks regarded the name as non-European. Although not a strict definition of ethnic background, this is the definition most relevant to the potential discrimination under investigation. For each applicant the following data were recorded: order of application, non-European name, sex, scores for individual components of the assessment by both selectors, and final outcome of the application (offer or reject). When assessing forms blind, selectors were asked to indicate whether they had identified the applicant as being from an ethnic minority group from information provided on the form.

Analysis of data
Data were analysed in three stages with SPSS. Firstly, to confirm that applicants from ethnic groups receive lower marks, total score was modelled by linear regression. Secondly, differences between blind and open scores for each individual were compared to assess the effect of blinding. Finally, component scores were dichotomised at about the overall median so that numbers of candidates dropping more marks than average could be analysed for each component. Our interest was only in discordant scores, and we had no prior view as to whether blinded scores would be better or worse than open scores. Poisson regression of discordant scores was used to assess whether blinded scores were more worse (or less better) in the applicants from ethnic minority groups, which would indicate the putative discrimination.

	Results

Top Abstract Introduction Methods Results Discussion References

In total 2047 applications were included in the analysis, of which 1485 (72.5%) were adjudged to have a European name by the administrative staff. Overall, 166 (29%) forms (including two adjudged European) were identified as being from an applicant from an ethnic minority group by the selector assessing the anonymous form, whereas the remaining 398 were classified as unidentified ethnic minority group (table 1).

Association of ethnic group with total score
The mean (SD) of total score was 28.7 (4.6) points (fig 1). The corresponding figures for each group were: European 29.2 (4.3), unidentified ethnic minority 27.2 (5.1), and ethnic minority identified 27.7 (4.3). This group difference was highly statistically significant (P<0.0001) by linear regression with or without adjustment for potentially confounding factors (sex, time of application, combination of scorers). Early application and being female (difference 1.3, 95% confidence interval 0.9 to 1.7 points) were also both significantly associated with higher scores.

View larger version (28K): [in this window] [in a new window]   Fig 1.   Distribution of total scores at shortlising for all applicants for entry in 1998 according to study groups

View larger version (18K): [in this window] [in a new window]   Fig 2.   Comparison of scores awarded blind and open for each of subgroups defined by surname

Comparison of blind and open scoring
The differences between blind and open total scores did not significantly differ from zero in any of the three groups (fig 2). Using linear regression to control for confounding factors as above, the unidentified ethnic minority group had similar differences between blind and open scores to the European group (0.00, 0.20 to 0.20). The identified ethnic minority group had larger differences of blind minus open scores than the European group (0.25, 0.04 to 0.53), points compatible with minor positive discrimination, but this was not statistically significant (P=0.09).

Analysis of component scores
Analysis of the components of the blind score showed the identified ethnic minority group to be less likely than the other groups to drop marks for the career insight component (table 2). In each of the other components the pattern was that the European candidates were least likely to drop marks, with the identified and unidentified ethnic minority groups performing similarly.

	Discussion

Top Abstract Introduction Methods Results Discussion References

Defining ethnicity
Classification by non-European name is not identical to ethnicity (table 4). However, if the lesser success of applicants from ethnic minority groups at shortlisting is due to discrimination then the name is the clearest marker of ethnicity available to the selector and has been shown previously to predict shortlisting outcome.⁵

Outcome for ethnic minority applicants
In keeping with previous data both nationally and at Leeds school of medicine, applicants from ethnic minority groups scored less well than European applicants. Scores peaked just below 31-32 points, which for entry in 1998 was where the division between reject and interview and offer occurred. Thus the observed mean difference of just 1.5 to 2 points influenced the outcome for a large number of applicants.

Anonymising forms
Making application forms anonymous proved difficult and required a photocopy of already copied applications, with a further reduction in legibility of some forms. The applicant's name appears at least three times on each form and often up to 15 times, so complete removal required close scrutiny that was time consuming.

Effects of anonymous shortlisting
Even when application forms were successfully anonymised the lack of difference between open and blinded overall scores suggested that disadvantage did not result from direct discrimination by selectors. The only statistically significant difference between blind and open assessment on any component of the total score pointed to positive discrimination, with more than anticipated of the ethnic minority group identified having better blind than open scores. This observation, on one quite subjective component, led to only a 0.25 (0.04 to 0.53) difference in total score: not sufficient to make a practical difference. Nevertheless, any form of discrimination, whether positive or negative, is highly undesirable and offers another reason why anonymising applications is detrimental to the fairness of the selection process. Admissions tutors were aware of this study, and this finding may indicate a heightened awareness by the selectors that their performance with respect to racial discrimination was being assessed. It is possible that the selectors may also, for the same reason, have avoided negative discrimination during the study period, although these behaviours are clearly difficult to test.