Career focus

New approaches to selecting medical staff

Last week I noted that traditional approaches to recruiting doctors often did not define what was being looked for in medical staff and used methods of poor validity such as the letter of reference and the unstructured interview.1 In this paper I review the use of other methods, which have been devised mostly by industrial and organisational psychologists and may give better results.

Psychological tests
Extensive research on intelligence tests shows that they predict performance at work well, for nearly all types of work. They may, however, be less useful in selecting doctors, given that access to the profession has depended on very high grades in university admission examinations. These entry requirements are likely to mean that few doctors would get anything but high scores in intelligence tests, so such tests would not contribute useful information in selection. The American general aptitude test battery has information on average intelligence quotients (IQs) for 444 occupational groups, in which general practitioners (with an average IQ of 136) are beaten to the top only by mathematicians (mean IQ 143).2

Some approaches to testing intellectual ability favour aptitude batteries - from four to 12 separate tests, assessing spatial ability, memory, manual dexter- ity, etc. In theory, differential aptitude testing should give more accurate predictions on the assumption that each job requires a different profile of abilities: surgeons need dexter- ity, general practitioners (these days) need financial numeracy, psychiatrists need verbal skills, etc. In practice, scores for different aptitudes are highly correlated, so multiplying the number of tests may not add that much extra information.3 A high score in an ability test does not guarantee that the applicant will not be lazy, disruptive, lacking in social skills, or poorly adjusted. Intellectual ability is necessary but not sufficient for success at work. Tests of personality, assessment centres, biographical methods, or structured interviews are needed to complement tests of intellectual ability and may be more important for medical work than for most other work.

Scores on CPI for medical students T score* Achievement via independence 65 Psychological mindedness 61 Empathy 60 Dominance 59 Self control 54 Amicability 54 Good impression 53 CPI=California psychological inventory *The mean is set at 50 and the SD at 10, so a score of 60 is one SD above the mean and puts a subject in the top 13% of the general population for that characteristic

Personality tests
Personality tests are more controversial than tests of intellec- tual ability. Some critics claim that they are an intrusion. Other critics claim that they simply do not work. Personality test tends to mean, in practice, personality questionnaire; other methods such as Rorschach inkblots are cumbersome and ineffective as selection tests. The way the items are chosen distinguishes a proper personality questionnaire from a list of questions thrown together as for a magazine quiz entitled "Are you a caring person?" Most good questionnaires are empirically keyed using criterion groups. A dominance scale is derived from answers given by people known to be good at leading and influencing others.

There are a dozen theoretical approaches to personality, and as many models of how many aspects of personality are worth measuring. Space permits only four examples. The California psychological inventory includes 462 questions to measure 22 "folk concepts," including dominance, sociability, self control, responsibility, flex- ibility, and three facets of ambition. This inventory has been used in selecting medical students in Australia.4

The 16PF questionnaire, as its name implies, measures 16 personality factors and has also been used to select medical students in Malaysia; candidates who are more reserved, less emotionally stable, and more apprehensive are more likely to have problems in medical school.

The Myers Briggs type indicator measures four bipolar dimensions of personality relating to how people take in information and make decisions. It is particularly suitable for career development and team building. Groups of people who all have the same outlook may get on well together and find each other congenial, but they may tackle problems in only one way and see things from only one perspective, which makes them vulnerable in a world of incessant change. A more effective team will usually contain a mixture of different types as measured with the indicator. The strong interest inventory has 209 occupational interest scales, which compare a person's answers with keys derived from criterion groups of people successful in various vocations. The healthcare sector is well represented: physician, two levels of nurse, administrator, ambulance crew, and a range of paramedi- cal staff.

Criticism of personality questionnaires focuses on their validity (their alleged inability to predict work performance) and the problem of faking good answers - answers that a person hopes will secure the job rather than answers that really describe him or her.

Personality of medical students
There are as yet few or no data on the personality profiles of medical staff in the United Kingdom. However, data on 551 male American medical students who completed the California psychological inventory yield several interesting facts (box). Overall, these data show that:

	Medical students score above the general average on all scales, high scores being "better"
	Their highest score is for achievement via independence, the desire to get ahead by one's own efforts
	Their lowest scores are for self control, the desire to create a good impression on others, and amicability, although all three are still higher than the average in the general population
	They are not lacking in interest in others (psychological mindedness) or in the ability to read others and understand their points of view (empathy), tending to disprove the claim that medical schools recruit or produce cold, unfeeling people.

Recent reviews show that personality tests are only moderately successful in predicting proficiency in a job, but they are much better in predicting commendable behaviour, delin- quency and substance abuse. Commendable behaviour is defined by letters of recommendation, letters of reprimand, disciplinary actions, demotions, involuntary terminations, and ratings of effort and hard work. Delinquency means theft, conviction, or imprisonment. Substance abuse covers alcohol as well as illegal drugs. Researchers in the United States armed forces have compared the prediction of technical proficiency and general proficiency with the prediction of effort and leadership and personal discipline in both personality tests and intellectual ability tests. They found that personality questionnaires predict motivational aspects of work performance ("will do") better than general and technical proficiency ("can do") and that personality questionnaires have incremental validity - that is, they predict aspects of work performance not predicted by tests of intellectual ability.6

Personality questionnaires are often criticised as transparent and easy to fake. No one applying for a surgical post is likely to agree "I don't like the sight of blood" (an item from the first ever personality test, used to screen army recruits in 1917). Faking may be deliberate lying or half conscious distortion.

A questionnaire necessarily measures a person's self concept, which in well adjusted people is usually fairly favourable. Those who use tests may be unduly concerned about faking. Recent analyses comparing test validity in candidates who tried hard to present themselves in a good light and in those who did not found that attempting to fake good results does not reduce the validity of the personality test.7 By their very nature, personality questionnaires tend to ask questions that seem personal to some candidates - for example, "Are you often troubled by feelings of self doubt?" The tester should explain why the test is being used and what will be done with the data. Candidates should always be given the option of not answering any question that they find intrusive.

Assessment centres
An assessment centre is not a place: it is a combination of different assessment methods, with an emphasis on group activity. For example, a typical assessment centre programme to appoint a hospital manager might last a day and include a personality questionnaire, an in tray exercise, ability tests, two group exercises, and a presentation. Assessment centres work on the principle of multi-trait, multi-method assessment. Any single assessment method may give misleading results: some people interview well, while others are good at tests.

Having decided what dimensions of work performance are to be assessed, the assessment centre planners select or devise at least two, qualitatively different methods of assessing each dimension. Empathy is assessed by group exercise and personality inventory, while numerical ability is assessed by financial case study and timed numerical reasoning test. An assessment centre that does not have a matrix plan is not a real assessment centre. Unfortunately, too many people think that an assessment centre is any collection of tests and exercises - begged, borrowed, or stolen - and included for availability, not because they are accurate measures of important dimensions of work performance.

Assessment centres use a variety of group and individual exercises. In assigned role exercises each person has a brief such as to compete for a share of a single budget or push a candidate for a job. These exercises are used to assess negotiating skills, persuasiveness, and the ability to compromise. In unassigned role exercises candidates have a simulated business to run in which decisions must be made rapidly, with incomplete information, and under constantly changing conditions. They are used to assess tolerance of uncertainty, tolerance of stress, and flexibility.

In leaderless group discussions candidates have a task to complete such as agreeing a plan or a set of priorities, but no one is nominated to chair the meeting. These exercises are used to assess the ability to influence others and the ability to complete a task inside a time limit.

In team exercises candidates are split into two opposing groups; one group collectively advocates one side of a case and the other half takes the opposing view point. These exercises are used to assess negotiating skills, teamwork, analytical skills, and problem solving ability.

In role plays the candidate handles a dissatisfied patient or an employee with a grievance. Role playing is used to assess communication, listening, and interpersonal skills.

Several reviews find assessment centres achieve generally good results. A 30 year follow up of the Civil Service Selection Board, a long running assessment centre for senior civil servants in Britain, showed it to be one of the most accurate selection programmes ever recorded.8 Critics have argued that assessment centres answer the question: "Does his or her face fit?" better than: "Can he or she do the job?"9 This implies that assessment centres fill the organisation with carbon copies of the present senior staff, if not with yes men and sycophants. Selecting clones of present staff often is a bad idea but is probably better than filling vacancies at random, which is all some methods can achieve.

Mark Cook,
Personnel Selection Research Group,
University of Wales-Swansea,
Swansea SA2 8PP

Further reading
Cook M. Personnel selection: adding value through people. London: Wiley, 1998.

References
1 Cook M. Traditional ways of selecting medical staff.career focus. BMJ 1998; 316 classified suppl:2-3.(7 March)

2 United States Department of Labor. Manual for the USTES General Aptitude Test Battery. Section III. Development. Washington, DC: US Department of Labor, 1970.

3 Ree MJ, Earles JA. Predicting training success: not much more than g. Personnel Psychology 1991;44:321-32.

4 Tutton PJM. Medical school entrants - semi-structure interview ratings, prior scholastic achievement and personality profiles. Med Educ 1993;27:328-36.

5 Peng R, Khaw HH, Edariah AB. Personality and performance of pre-clinical medical students. Med Educ 1995;29:283-8.

6 Hough LM. The "big five" personality variables construct confusion: description versus prediction. Human Performance 1992;5:139-55.

7 Hough LM, Eaton NK, Dunnette MD, Kamp JD, McCloy RA. Criterion-related validities of personality constructs and the effect of response distortion on those validities. J Appl Psychology 1990;75:581-95.

8 Anstey E. A 30-year follow-up of the CSSB procedure, with lessons for the future. J Occup Psychology 1977;50:149-59.

9 Klimoski RJ, Strickland WJ. Assessment centers - valid or merely prescient? Personnel Psychology 1977;30:353-61.

Briefing
	Cambridge medical graduates are an atypical group in some respects: older, less likely to be female, in possession of a first degree, and having studied on a shorter than usual clinical course, but the results of a 15 year follow up survey of the university's graduates make for some interesting reading (J R Coll Phys Lon 1998;32:49-55). The graduates' career choices range across the specialties, though only about a quarter opt for general practice, and more than 10% are in academic posts. Almost a fifth of a sub- sample of graduates between 1991 and 1993 were still undecided on eventual career choice. Of 1002 respondents, fewer than 1 in 10 worked part time, 59 were working abroad, and only 24 had left medicine altogether, though this group may be overrepresented in the fifth of graduates who did not respond.
	Moving to asynchronous communication (voicemail or email rather than conversation) may be more efficient, but a recent survey reported in Employee Health Bulletin (1998;1:12-14) found that just over half of a sample of regular email users had received abusive email ("flaming"). This wastes time as the recipient typically seeks immediate counselling from colleagues or, in about a third of cases, crafts an equally aggressive reply. Progressive organisations will have an email policy that includes a reminder that email is not a substitute for personal face to face communication.
	Got a career question? Email advice is readily available from fellow subscribers to the career-f mailing list. Just follow the instructions in the box below.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?