“Longlisting” from application forms

Equal opportunities

Postgraduate deans are legally obliged to ensure equal opportunities in specialist registrar appointments. Surprisingly, this does not mean that they have to ensure that the best candidate is picked. To do so would be very costly, as it would require highly trained recruitment teams and lengthy screening and appointment procedures.

What they do have to ensure, however, is that the processes used in selection could not have created bias against a candidate or group of candidates.(1) The procedure has to be fair to all. Of course, you could ensure no sexual or racial prejudice by picking candidates at random. However, that would not be fair to those who could demonstrate better suitability for the post. Trying to select the most suitable candidate sounds a good way of being fair, if you knew what comprised suitability and how an applicant might demonstrate it. Real equal opportunities, then, is to have a system that attempts to identify the ideal and which rigorously avoids bias in comparing candidates with that ideal blueprint. This blueprint is called the person specification and often is divided into those features that are essential and those that are desirable.

Does “equal opportunities” work?

The short answer must be no. Even assuming that a person specification could be phrased in such a way as to crystallise perfectly the concept of the ideal, it is difficult to imagine how this could be made to work. Is it possible to make an unbiased comparison of every curriculum vitae with that ideal in mind, and a subsequent prioritisation of merit? By the 300th curriculum vitae, will a shortlister still be efficiently examining candidates' understanding of audit, appropriate experience, etc?

Some people genuinely do sit with the person specification in front of them and diligently check each feature. This means getting through only about 10-12 curricula vitae an hour (personal observation). This meticulous process can be made more scrupulously exact by the addition of a scoring system, such as the one devised in North West Thames for obstetrics and gynaecology specialist registrar posts. Such a system not only takes extra time to complete but, more worryingly, is difficult to interpret by having to add marks together to make a total score. For example, is a fast tracking senior house officer with two years' experience (4 points) and five publications (3 points) equal to a sluggish senior house officer with three years' experience (6 points) and some informal teaching of students (1 point)? A scoring system is admittedly unbiased, but it is time consuming and inflexible and does not necessarily pick the best candidate.

“Failure to shortlist” codes

A final nail in the coffin of traditional shortlisting is the insistence of postgraduate deans on allocation of a “failure to shortlist” code for unsuccessful candidates. A scoring system answers this need, but has the drawbacks mentioned. On the other hand, longhand explanation of where a candidate fell down may leave the issue open to dispute and would be time consuming.

Is there a way to reduce the time of shortlisting, increase its accuracy, keep it unbiased, and give coded information on failure to shortlist? Perhaps “longlisting” could be the answer.

Longlisting

It is apparent to most shortlisters that many candidates' curricula vitae do not come up to scratch in key areas such as achievement, progress, and quality of communication. Much time can be wasted on sifting through and scoring such curricula. The concept of longlisting is the sifting out of those who clearly do not match up to what is really wanted. The shortlist is derived by discussion of the remainder. In order to achieve this, the traditional person specification must be discarded and replaced with one that reflects exactly what a shortlister is looking for. This is achieved by rewriting the person specification into usable categories (see box).

The person specification gives a cameo of what is required in each category. These categories are then arranged on a grid, and the assessor is asked simply to tick if he or she feels that a candidate meets each criterion (table 1). If the candidate fails an essential criterion the assessor immediately moves on to the next applicant. A cross in the appropriate column denotes the reason for “failure to shortlist.”

Application form

This process is much facilitated by having the desired information in curricula vitae arranged in the same order as the categories in the assessment grid. An application form can readily achieve this. Furthermore, the application form can contain space for the category “Supporting information,” in which candidates can outline why the post is right for them. This section can also test communication skills, and hand written forms perhaps do this best.

Assessors are sent only application forms, with no curricula vitae. The idea is that, in this objective first sifting, assessors will come up with roughly the same “longlist,” from which a shortlist can be derived. Table 2 shows the results of a pilot study of this assessment method in five appointment panels. In about three quarters of cases the assessors all agreed on either inclusion or exclusion. Those excluded already had codes allocated by longlisting and so needed no further discussion. Applicants who were “mainly rejected” could have been discarded automatically if the system was proved. As it was, they were all discussed, and none was shortlisted at any of the panels. The candidates listed “for discussion” (as to whether they should join the shortlist) usually represented well under 10% of the applicants.

The time taken to review application forms was measured and was remarkably constant for each panellist, at about 1 minute per form. The time taken for the shortlisting meeting was greatly reduced as only a few candidates needed to be discussed.

Conclusion

Longlisting by an equal opportunities application form and a tick fist is a new approach to selecting candidates for a post. The pilot study showed that it was considerably quicker than traditional shortlisting and that it is likely to prove more valid and more reliable. Further work is being done on reliability testing and on anonymity, but, meanwhile, I recommend this process as a step forward in equal opportunities recruitment.