Traditional ways of selecting medical staffBMJ 1998; 316 doi: https://doi.org/10.1136/bmj.316.7133.2 (Published 07 March 1998) Cite this as: BMJ 1998;316:S2-7133
In the first of two articles on recruiting doctors occupational psychologist Mark Cook examines traditional methods of recruiting doctors and finds them wanting
The crucial first stage in assessing staff is deciding what you are looking for. The traditional methods of job description and person specification do not dig very deep. Specialised job analysis techniques give more useful results and make it much easier to defend selection methods against criticism or fair employment challenges. A recent analysis of the work of hospital consultants in the Netherlands listed a dozen or more themes, of which “medical competence” was only one.
Other important attributes included being open to criticism, staying calm in a crisis, a caring attitude to patients and relatives, willingness to take a share of the non-medical side of the work, and being cost conscious.1 Research in Israel suggests that compassion and empathy are what patients look for in doctors but do not necessarily find because neither qual- ity seems to help doctors advance their careers in hospital.2 American research confirms that effective performance as a medical practitioner has two basic dimensions: medical skills and management of psycho- social aspects of illness - that is, talking to, listening to, and re- assuring patients.3
In Britain the past 10 years have seen doctors increasingly becoming also managers, whether formally, as practice managers or clinical directors, or informally. Most professional people also need a range of skills besides their professional expertise - for example, making presentations, writing reports, understanding budgets, using computers, dealing with staff, and chairing meetings.
In Britain medical staff are usually selected by the traditional combination of application form, reference, and interview. Previously they will have been selected for medical school by application form, reference, and examination grades. In America entry to medical school also depends on getting a very high score in the medical college admission test, which predicts grades in medical school slightly better than do examination results at school.4 The admission test assesses general intellectual ability, in the shape of problem solving and analytical skills, as well as knowledge of biology, chemistry, and physics.
The selectors' first task is to sift out the few applications that will be taken further. Recent research on admission to medical school in Britain found that candidates with non-European surnames were more likely to be rejected at this stage.5 Similar results have been reported in the United States, where non-American applicants were, other things being exactly equal, less likely to be offered residency training positions in psychiatry.6 Sifting of applications can be made less arbitrary by using training and experience ratings, which assign points to each application using an agreed marking scheme (but do not refer to sex or ethnic group).7 More elaborately, past applications can be analysed to identify factors associated with success, which can be scored in future applications to sift out those more likely to succeed. This technique requires very large numbers and careful analysis. It is expensive to install, but once in place it is cheap to operate and invisible to applicants.
The second hurdle is usually the reference. In Britain requests for references are usually open ended, saying in effect: “Tell me what you think of J Smith, in your own words.” In the United States, however, ratings and checklists are more usual. Reference requests do not generate much useful information. Many reference writers use a private language of damning with faint praise or expressing coded reservations, which creates considerable scope for misunderstanding. Analysis of letters of reference for American medical school applicants shows that they reveal more about their authors than they do about the candidates.
In one study on medical school records, researchers unearthed 20 cases in which two referees (A and B) had written references for the same two candidates (X and Y).8 If A and B are to be potentially useful sources of information, then they should tend to agree with each other about each candidate. In fact, what A said about candidate X tended to agree with what A said about candidate Y. In other words, the reference tells you more about the person who wrote it than about the person it is supposed to be describing, which suggests references may not be very useful. References have one thing in their favour - they are cheap to collect.
The third hurdle is the interview. Nearly all British employers interview shortlisted applicants, and most still base their decisions largely on interview performance. Reviews of interview research find that conventional one to one interviews achieve the lowest validity and are not much better than choosing at random.9 Conventional board or panel interviews achieve slightly better results than one to one interviews but are still very inaccurate.10 This implies that the interview as generally observed in British practice may be a poor way of assessing staff. The slight superiority of panel interviews vindicates the healthcare sector's traditional use of them.
Structured interviews are very much more successful and achieve some of the best results of any selection methods. However, structured interviewing does not mean following a seven point plan or agreeing who asks what before the interview starts. It means devising an entire interview system in which every part is specified in minute detail. Interviewers' questions are structured, often to the point of being tightly scripted. Interviewers' judgments are structured by rating scales, checklists, etc. Candidates' answers are compared against sets of model answers. Structured interviews work well but are expensive to set up and tend to be job specific. At their most structured, structured interviews blur the distinction between interview and written test. Why not read the questions to a group of interviewees, or even print them as a questionnaire?
Other selection procedures
Examinations at medical school have traditionally included work sample tests - candidates are given patients to examine and diagnose. Work samples have several big advantages: they achieve good validity, they are fair, and they look fair. They are obviously closely job related and avoid any problematic inferences from intellectual ability, speed of learning, etc. Work samples have some disadvant- ages. They are highly specific to the job, so medical selection might require several dozen; they require equipment and props, which may take up a lot of room; and they can test only one person at a time, which makes testing expensive and slow. Work samples are best suited to jobs with several fairly concrete tasks, but they do not work so well for jobs with many diverse tasks or with vague and unstructured tasks. This implies that they would have considerable scope for some aspects of medical work but would not give a complete picture by themselves.
Medical school has also traditionally included a strong element of what personnel specialists call realistic job preview, or showing applicants the less pleasant side of the work, in the form of extensive practice in dissection. Critics argue that too much time is spent on dissection and that it creates emotional isolation and aloofness in doctors.11 Graphology is widely used for selection in France. If handwriting does accurately reflect personality it would make a very cost effective selection method, because candidates could be assessed from their application forms. Neither is it obviously absurd to suppose handwriting reflects personality. However, all the available evidence indicates that graphology is not a good assessment method.12 Agreement between graphologists analysing the same handwriting sample is poor. Graphologists' ratings are completely unrelated to work performance. Graphologists often ask subjects to write pen pictures of themselves, so their assessments are not based solely on handwriting.
In the United States testing applicants for drug use is popular and controversial. Its advocates point to research in the United States Postal Service showing that drug use predicts higher absenteeism and higher involuntary turnover. The implication is that not employing drug users will increase productivity. Drug misuse has been identified as a problem for a few doctors in a recent British survey. Critics argue, however, that the size of the difference in work performance is tiny, far too small to justify the intrusion on the applicant's privacy and civil rights.13 Acceptability of drug testing depends on perceptions of danger, so the general public thinks it justified for surgeons, police officers, or airline pilots but not for caretakers, farm workers, or clerks. This brief review suggests that the selection of medical staff may not be highly effective because people do not always clearly define what they are looking for and then do not use effective methods to try to select people with those characteristics. The traditional trio of application, reference, and interview is known not to work very well.
Cook M. Personnel selection: adding value through people. Chichester: Wiley, 1988.
Cronbach LJ. Essentials of psychological testing. 5th ed. New York: Harper and Row, 1990.
Herriot P. Assessment and selection in organisations. Chichester: Wiley, 1989.
&Many books that purport to give prospective doctors an idea of what it is like to attend medical school give either a partial, official view, or a personal and idiosyncratic one (often with an American bias), but it would seem that it is the anthropologists who have the tools to get us past this difficulty. Making doctors: an institutional apprenticeship combines recognisable descriptions of life in medical school with a theoretical framework that does seem to have explanatory value for the many phenomena that constitute medical training.
The author, a psychiatrist who later studied anthropology, spent a year doing field work in a London medical school: his account explains the stability of the medical system and the educational and psychological problems that it poses.
Student culture is based on the acquisition of general professional dispositions - knowledge, idealism, co-operation, status, economy, and experience, played out on the “official” and “unofficial” back and front stages of medical school - its lecture theatres, rag weeks, revues, and the bar.
It provides a model for the subsequent high psychological morbidity of doctors: students learn how distress is perceived “backstage” (a range of reaction from amusement to frank distaste) and are encouraged instead to suppress it with readily available alcohol.
Do new curricular developments mean Sinclair's study is recording the dying moments of traditional professional training? The unchanging oral catechism traced here through Smollett's Roderick Random (1748), Gordon's Doctor in the House (1952), and present day ward rounds suggests not.
Sinclair S. Making doctors: an institutional apprenticeship. Oxford: Berg, 1997.