Research

Standardising the process versus improving the methods

BMJ 2006; 332 doi: http://dx.doi.org/10.1136/bmj.38776.497627.55 (Published 27 April 2006) Cite this as: BMJ 2006;332:1008
  1. Geoffrey R Norman (norman{at}mcmaster.ca), assistant dean, program for educational research and development1
  1. 1 Michael DeGroote Centre for Learning (MDCL), McMaster University, Hamilton, ON, Canada L8N 3Z5
  • Accepted 22 February 2006

Parry and colleagues surveyed 22 English medical schools to determine their approach to admissions.1 Their major concern is the lack of uniformity across schools, and their solution is the implementation of a centralised admissions process.

This may make good economic sense as there is little point in individual schools trying, for example, to create a standardised cognitive test to supplement or replace A levels. But if the goal of centralisation is simply to create greater uniformity, I think the effort may be misspent. In reviewing their paper, my sense is that there is, in fact, more commonality that difference across schools within England, and, for that matter, in most countries, where a combination of grades and some kind of “non-cognitive” measure is the norm. I am not particularly concerned with the individual variations they identified.

I do, however, have concerns about the admissions process, though these are different from those of the authors. I think we all have the cognitive-academic component pretty well in hand. While grade inflation, whether in A levels in the United Kingdom or undergraduate GPA (grade point average) in Canada, is a concern (the average GPA at McMaster last year was 3.74/4) the “fix” is straightforward, through the development of some standardised cognitive tests like the MCAT in the United States.

But I do not think that we are doing an acceptable job on the “non-cognitive” side, and I do not think it is simply an issue of training and standardisation of interviews. Measures like the personal interview are not nearly as defensible as the authors claim.2 One interview, like one patient case, is a seriously restricted sample, an insight that led directly to new approaches like the multiple mini-interview.3 Some believe that personality measures are a viable alternative to interviews, but in my view the evidence of reliability and validity to date is pretty inconclusive.4

So where do we go from here? It seems to me that some diversity of selection is expected and desirable because of the different educational cultures in different schools. I cannot see any compelling reason to insist on further uniformity of selection. After all, the standard assessment of student outcomes will ensure that the products are similar enough. We must do a better job of assessing the non-cognitive domain. I have no illusions that a better admissions process will identify all the potential “bad apples,” as some reformers hope. But we can all agree that both cognition and compassion matter and both should be assessed equally and well.

Footnotes

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
View Abstract