This is a preview. To continue reading please log in or Register to read this article

Shortlisting job candidates

Author: Alison Clayton-Smith

Summary

  • Shortlisting involves the employer reviewing applications against a set of job-related or shortlisting criteria and identifying a suitable pool of candidates for further assessment. (See What is shortlisting?)
  • There are several reasons for effective shortlisting, for example avoiding discrimination and ensuring a pool of good quality candidates. (See The business case for effective shortlisting)
  • To ensure that the shortlisting process is fair and objective, the employer should consider each application against the same set of criteria. These criteria should relate to the requirements of the job. (See The importance of developing shortlisting criteria)
  • The employer should develop shortlisting criteria by referring to the job description, competency profile and person specification for the role. (See Developing shortlisting criteria)
  • The employer should create a shortlisting panel of at least two people who will decide on the shortlisting criteria. (See Who should decide on the shortlisting criteria?)
  • Screening involves an initial review of applications against a set of basic qualifying criteria. (See Screening out unsuitable candidates)
  • The employer may use an online applicant tracking system to automate screening and/or shortlisting. This can save the employer a lot of time, but there are disadvantages to such systems. (See Online sifting)
  • Each member of the shortlisting panel should assess applications on his or her own to help prevent bias, following which they should meet to agree the final shortlist. (See The shortlisting process)
  • The shortlisting panel should avoid making assumptions about a candidate's ability to do the job and base decisions on information relevant to the role. (See Avoiding bias and discrimination in shortlisting)

Overview

This section of the XpertHR Good practice manual discusses the steps that employers should take to ensure that they carry out effective shortlisting as part of the recruitment and selection process. It covers: the importance of using shortlisting criteria; the use of screening; online approaches to sifting applications; the use of forms and scoring systems; the process of deciding whether or not to shortlist a candidate; and avoiding bias and discrimination during the shortlisting process.

What is shortlisting?

Shortlisting forms the first of two stages in a selection process to choose the best candidate for the job. Shortlisting involves the employer reviewing application forms and/or CVs and identifying a final list of candidates for the second stage, when typically assessment by interviews and/or tests takes place.

Usually, the employer carries out a screening process before it shortlists candidates for selection. Screening involves filtering out applicants who do not meet the basic qualifying criteria for the post, for example one basic qualifying criterion for a vacancy in HR could be the possession of a CIPD professional qualification. The employer can conduct a screening exercise either by manually sorting through the applications for a vacancy or by carrying out an online screening process or conducting telephone interviews. An online screening process can take a number of forms, for example a brief questionniare asking the candidate basic qualifying questions about his or her suitability for the post, or a more sophisticated software tool that screens out candidates who do not have the skills for the role, or a personality test.

Shortlisting is a more in-depth process, whereby the employer assesses applicants against the essential and desirable criteria for the role and decides which individuals should progress to the next stage in the selection process. Following a screening process, the employer could still have a potentially large pool of candidates who meet the basic qualifying criteria, and the aim of the shortlisting exercise should be to reduce this pool to a manageable field of candidates for selection.

Not all employers differentiate in practice between screening and shortlisting. Sometimes, the employer carries out screening and shortlisting simultaneously, rather than as two distinct stages. This approach is possible where the original pool of applicants is small; if there is a large volume of applicants, it is good practice for the employer to conduct a screening process to check applicants' basic qualifying criteria before proceeding to a more detailed shortlisting exercise.

As part of the recruitment process, individuals who are interested in a vacancy typically submit an application form or CV. Sometimes, recruiting organisations require additional evidence of an applicant's suitability for the post, for example the completion of screening tests or personality questionnaires. To identify a shortlist, the employer should review all of the data obtained on applicants at this application or screening stage against a set of shortlisting or job-related criteria. At the end of this process, the employer should place the candidates who most closely meet the essential and desirable shortlisting criteria on the shortlist. The shortlisted candidates will progress to the next stage of the selection process.

The screening and shortlisting processes can be automated using specialist software that scans applications for key words relating to the criteria.

The business case for effective shortlisting

There are four key reasons for having an effective shortlisting process:

  • Legal implications: An organisation can incur significant costs if a candidate makes a tribunal claim that he or she was not shortlisted on the basis of discrimination. For example, failing to shortlist an applicant with dyslexia because there were spelling errors on his or her application where the role does not require good written communication skills could leave the employer vulnerable to a claim. Under data protection law, all applicants have the right to see any data held about them from any part of the recruitment and selection process, including shortlisting decisions.
  • Quality of candidates: Having shortlisting criteria that are too stringent could mean that the employer misses out on candidates who may be suitable for the role.
  • Time and cost: Applying shortlisting criteria that are not stringent enough could mean that the employer includes candidates on the shortlist who are not suitable for the role, wasting time and resources at the next selection stage.
  • Employer reputation: The recruitment and selection process can have a significant impact on the employer brand. The recruiting organisation should treat applicants fairly at every stage of the process so that they have a positive impression of the organisation, regardless of whether or not they are shortlisted for the post.

The importance of developing shortlisting criteria

To ensure that it identifies the most suitable pool of candidates for the vacancy, the employer should develop shortlisting criteria, also referred to as job-related criteria, for the purposes of shortlisting. These criteria will enable the employer to match the skills and competencies of applicants against the requirements of the job during the shortlisting process, so that the best candidates go through to the next stage of the recruitment process. Each role should have its own set of shortlisting criteria.

Another reason for using shortlisting criteria is that, if the organisation has to justify its shortlisting decisions, it should be able to demonstrate that it followed an objective and consistent shortlisting process. This means that the employer should shortlist all job applicants against the same set of criteria.

Developing shortlisting criteria

To develop shortlisting criteria (also called job-related criteria), the employer should refer to the job description, person specification and competency profile for the role. These documents should indicate the essential and desirable criteria needed to be able to do the job and provide a minimum level that shortlisted candidates need to reach. The following are examples of shortlisting categories:

  • Level of educational qualification or equivalent: For example, the employer could advertise a graduate position that requires the candidate to hold a 2:1 minimum degree.
  • Type of experience: For example, the employer could specify that a candidate for a secretarial role needs experience in taking and writing up minutes.
  • Skills needed: For example, an HR role could require experience in using a certain type of HR database.
  • Knowledge needed: For example, the employer could advertise a social media role as needing knowledge of a range of social media tools.
  • Behavioural competencies: For example, the employer could ask for evidence of influencing at board level when recruiting to an accountancy role.

Essential criteria should be job requirements that a candidate must satisfy to do the job. The employer's expectations can be too high if it sets criteria for the "perfect" candidate at the shortlisting stage, for example asking for a Master's degree when this qualification is not necessary to perform well in the role. If the employer sets unrealistically high shortlisting criteria, it may filter out potentially suitable candidates and be left with a very small pool of shortlisted candidates.

The employer can establish "desirable criteria" as a way of distinguishing between candidates who meet only the essential criteria for the role and those who offer additional relevant qualities. If a candidate meets both the essential and the desirable criteria for the role, this could represent the difference between a satisfactory employee (who meets only the essential criteria) and a high-performing employee (who meets both sets of criteria). Whether or not the employer applies the desirable criteria to the shortlisting stage depends on the number and quality of applications that it receives for a vacancy; typically, the employer will shortlist all of the candidates who meet the essential criteria. However, if the vacancy attracts a high number of applications that meet the essential criteria and it needs to reduce the number of shortlisted candidates, one option is for the employer to assess candidates against the desirable, as well as the essential, criteria for the role.

Who should decide on the shortlisting criteria?

The employer should involve more than one person in deciding on the criteria that it will use to shortlist candidates. This helps to limit the potential for personal bias to have an impact on the criteria, and the discussions involved in identifying the criteria provide a useful way of identifying "essential" versus "desirable" criteria.

Typically, a shortlisting panel includes the recruiting line manager and an HR representative. The employer could involve other members of staff in the process depending on:

  • the importance or sensitivity of the role;
  • other managers' understanding of the role and the contribution that they could make to the shortlisting process; and
  • the need to gain buy-in to the selection process from other employees.

For example, the recruiting manager might decide that a team member in a similar role could provide useful input and should be included on the panel responsible for developing the shortlisting criteria. This approach has the added benefit of providing a developmental opportunity for the team member. Another example is where a role will have a dotted reporting line into another department, for example if the postholder will be accountable to the IT department for certain projects but will work within the finance department and be line managed by a finance manager. In this example, the employer could include an IT representative and the finance manager on the panel.

Screening out unsuitable candidates

If there is a small number of applications, typically the employer will undertake one initial selection stage and shortlist all candidates who meet the essential criteria for the role before proceeding to the next selection stage. If there is a large number of applications, the employer may undertake two initial selection processes: screening and shortlisting. In this situation, the employer should establish a "long list" of candidates to go forward to the more detailed shortlisting process.

Typically, screening or "sifting" enables the employer to screen out candidates who do not meet the basic qualifying criteria. The employer can carry out the screening process manually, by sifting through every application or CV that it receives. This approach can be time consuming and is more appropriate if there is not a large number of applications. Alternatively, if the employer anticipates receiving a large number of applications and it has the necessary resources, it can set up an online screening process. Typically, an online screening exercise requires the candidate to complete a brief questionnaire as part of the application process. The employer should base the questions in the online screening exercise on the basic qualifying criteria for the role.

Another screening option is for the employer to conduct brief telephone interviews with applicants. The employer is more likely to carry out telephone interviews with candidates as a second screening stage, after it has established whether or not candidates satisfy the basic qualifying criteria for the role. Telephone interviews can be time consuming and most employers cannot justify their use as an initial screening exercise.

Screening involves assessing applications against the shortlisting (or job-related) criteria. For example, the employer could ask itself the following questions when it carries out a manual screening process or designs a screening questionnaire for online or telephone use:

  • Has the candidate applied for the right job? Where the employer has advertised the vacancy on a commercial job board, it could be easy for candidates to submit an application for lots of jobs at the click of a button. This increases the risk of inappropriate applications that need to be filtered out before the shortlisting process.
  • Does the candidate have the required qualification(s)? For example, an applicant for an accountancy role needs to have the appropriate accountancy qualification(s).
  • Does the candidate have the required technical skills? For example, an IT programmer could need skills in specific IT programming software.
  • Does the candidate have the required industry experience or professional knowledge? For example, an applicant for a business development manager role could require knowledge of the recruiting organisation's industry.
  • Do the candidate's salary expectations match the advertised salary? There is little point in the employer shortlisting a candidate who would not accept a job with the stipulated salary.

Not all of the suggested questions are suitable for every vacancy, and the relevance of the questions will also depend on whether the employer is carrying out a manual, online or telephone screening exercise.

Online sifting

Online screening tools enable employers to reduce the size of the candidate pool for a vacancy by applying qualifying criteria for the role at an early stage of the application process. Typically, the recruiting organisation uses an online screening process as part of an electronic applicant tracking system (ATS) that receives online applications, screens out unsuitable applicants, compiles a shortlist and tracks candidates' progress through the recruitment and selection process.

As well as processing applications and communicating with candidates by sending out automatically generated emails, an ATS can administer a range of online tests designed to test candidates' suitability for the post. Online screening systems are most useful for volume recruitment, for example a graduate recruitment exercise where the employer wishes to hire a significant number of graduates and expects to receive hundreds of applications. The main advantage of using an online screening system is that, once the employer has developed an automated process, it can reduce a large number of applications to a manageable shortlist. In addition, an ATS can send out all relevant rejection emails and issue invitations for the next stage of the selection process. These functions can save the employer a significant amount of time and mean that the system, if developed properly, can draw up a shortlist quickly and efficiently. In theory, because all candidates go through the same process, without human judgment, online screening is an objective and fair way of developing a shortlist.

However, under the General Data Protection Regulation (2016/679 EU) (GDPR), individuals have the right not to be subject to a decision based solely on automated decision-making. The right applies where the decision would significantly affect the individual, which would cover a decision on whether or not to shortlist someone for employment.

There is an exception to this right where automated decision-making is "necessary" to enter into a contract. The guidelines on automated decision-making under the GDPR produced by the Article 29 Data Protection Working Party state that this exception could apply to allow solely automated shortlisting decisions if this is necessary due to an exceptionally high volume of applications for a vacancy (it gives the example of tens of thousands of applications). However, in most cases, it is unlikely that automated decision-making would be "necessary", as it will usually be possible to make the shortlisting decision with human intervention.

Even where the exception does apply, the employer can make shortlisting decisions on a solely automated basis only if it informs the individual of this and puts in place safeguards, including the right for the individual to require human intervention in the processing, to express a view and to contest the decision. The employer must ensure that there is a procedure in place to allow staff to review the decision and make a new assessment of whether or not to shortlist an individual, if he or she objects.

One of the main disadvantages of online screening systems is that the employer still needs to develop an appropriate system (either in-house or with the assistance of a specialist provider). This means that there is the potential for bias to creep into the development phase. To help prevent this, it is important for more than one person to take part in this phase. Further, the employer should establish a robust process for checking that the system accurately filters out candidates' applications against the shortlisting criteria. It is good practice for the employer to run a pilot exercise to test the screening system several times before launching it. To carry out a pilot exercise, the employer should use a pool of dummy candidates and verify whether or not the online screening system produces the same pool of successful candidates that is produced following a manual screening exercise. This pilot process should highlight any issues of concern around inappropriate or missing criteria within the screening tool.

There are further potential disadvantages of using an online screening or shortlisting system:

  • Applications are necessarily made online: This means that the system could penalise candidates who are not, and do not need to be, comfortable using technology.
  • Cost: Online systems vary in price and it can be expensive to develop a bespoke system, although there are free "open source" systems available on the internet.
  • Cheating: There could be applicants who are looking to "beat the system" by including certain key words in their application, even though these do not relate to their actual experience. It is important for the employer to remember that online systems are not foolproof.
  • Missing out on suitable candidates: There could be applicants who have the necessary experience and qualifications for the role, but do not use the exact words specified in the software. This could mean that suitable candidates are screened out of the selection process.

There are a number of online screening tools available to employers. These vary widely in terms of their design and sophistication (see table below). An ATS can automatically generate a simple, questionnaire-based form at an initial stage of the application process that verifies whether or not the candidate meets basic qualifying criteria such as having the legal right to work in the UK. For example, as part of its online application process for dealers, one casino uses a screening form that verifies whether or not candidates are aged 18 or over as this is a basic qualifying criterion for this role. Typically, this type of self-completion questionnaire is brief and should contain no more than 10 questions. If a candidate does not meet the basic qualifying criteria, the online system does not allow him or her to continue with the online application process.

Other screening tools are more sophisticated and can include personality questionnaires, online ability tests and work-sample exercises administered online. Some specialist providers can develop sophisticated work-sample and skills-based tests that rigorously test a candidate's suitability for the post. These types of test are based closely on the job-related criteria for the role and typically include a range of detailed questions that investigate a candidate's level of knowledge and skill in a specific area and how he or she would apply these in certain work situations. Some of these online tests are very sophisticated and can generate a question based on the candidate's previous response. These tests are rarely developed in-house by employers and are usually commissioned from specialist psychometric test providers.

One pre-screening tool that is available through some types of ATS is CV or application form matching functionality. This process allows the recruiting organisation to search candidates' applications for key words that reflect whether or not they meet the basic qualifying criteria for the role. For example, if a candidate for an HR position has used the words "CIPD qualification" in an online application and these match a basic qualifying criterion, the system would screen in the application to proceed to the next selection stage. However, when using such a tool, employers should be alert to the possibility that an applicant may have used a key search word in a context that would not indicate his or her suitability for the post. For example, the candidate may have stated that he or she does not currently hold a "CIPD qualification" but aims to study for one in the future.

Typically, recruiting organisations use online screening tools at an early stage of the application process, although the timing depends on the type of screening tool and vacancy. For example, the employer could incorporate a simple screening questionnaire designed to filter out candidates who do not meet the basic qualifying criteria for the role at the beginning of the application process. The employer could apply a more sophisticated screening tool that tests a candidate's skills and knowledge as a second screening process, following the receipt of applications for the vacancy or as the final application stage.

The following table considers the pros and cons of using specific options for online sifting. These options may be used on their own or in combination.

Pros and cons of options for online sifting
Option Pros Cons Example
CV or application form matching. Allows quick screening of CVs or application forms against key words.

The candidate may try to predict the key words and write his or her CV or application form to meet these, rather than submit an application that reflects his or her actual skills and experience.

Using key words could mean missing out on good candidates who have used different words that still reflect relevant skills and experience.

For candidates applying for an entry-level administrative role via an online job board, key words could include "Microsoft Office" and "time management".
Online questionnaire based on basic qualifying criteria for the role. Allows rapid sifting based on whether or not the candidate meets the basic qualifying criteria. If the form has not been developed properly, the process may screen out suitable candidates. A form for an accountancy position could include a question about whether or not the candidate has an accountancy qualification.
Self-test questionnaires and games Can be used to help potential candidates to decide whether or not it is worth their time applying for the vacancy.

Effectiveness depends on the self-awareness and honesty of the candidates. The test could act as a barrier if some potentially suitable candidates feel that the test is unfair or too time-consuming and decide not to apply.

The employer could ask potential applicants for a role in a warehouse about scenarios such as working indoors without daylight or their willingness to follow strict rules and procedures.
Online personality questionnaires. Provides a hypothesis about the candidate that can be explored at interview. Personality questionnaires depend on the self-awareness and honesty of the candidate. Therefore, questionnaires should not be used to sift people out in the early stages of selection. The employer could ask candidates applying for a customer service role to complete a personality questionnaire to assess whether or not they enjoy people-facing roles.
Online ability tests, for example a verbal reasoning exercise. Can be used to screen out applicants at an early screening or shortlisting stage by setting a minimum score to achieve based on the scores achieved by benchmark groups. Concerns about fraud, for example another individual completing the test on behalf of the candidate. One way to help overcome the problem is for the employer to administer a short retest in person at the next selection stage. The employer could ask candidates applying for a trainee IT programmer role to complete a test designed to assess logical thinking.
Work-sample and skills-based tests.

Can be designed to allow applicants to sample aspects of a role and given them an insight into the tasks that they could encounter in the role.

Can provide evidence of a candidate's ability to perform a real-life task.

It may take time to develop a test that is realistic, and it can be difficult to design one that can be marked online automatically.

The employer may require the candidate to do a short retest in person at the next selection stage to ensure there has been no fraud (for example, someone else may have performed the test on the candidate's behalf if the test is administered remotely).

The employer could ask candidates for a marketing role to complete a copywriting test that is marked via the use of key words and phrases.

Typically, the employer should develop a bespoke screening system for each vacancy, although it could adapt an existing one to suit a similar post. Whether the employer develops its own online screening tool or commissions a specialist external test provider to develop one will depend in part on the technology that supports the employer's careers site. The decision to develop screening tools in-house or procure their development from a contractor will also depend on the capability of the employer's corporate IT department. If the employer has an existing technologically advanced recruitment website and ATS, it could already have the necessary functionality to develop bespoke screening tools, particularly if they are simple forms or questionnaires to screen out candidates who do not meet the basic qualifying criteria for the vacancy. If the employer wishes to develop more sophisticated screening tools, it is more likely to need the expertise of a specialist test provider, although this option will be more costly. Another option is for the employer to use its own ATS for the online application process and transfer candidates via a hyperlink to an external test supplier's web portal that hosts the screening test(s).

The cost of a screening test depends on a number of factors, for example whether it is bespoke or off the shelf and its level of sophistication. The employer could receive a discount for the bulk purchase of a screening test, for example for a call centre vacancy it could cost around £25 per test to buy 500 single-use screening tests from a specialist test provider. To purchase a bespoke, sophisticated suite of screening tools for a specialist role could cost between £1,000 and £1,500. To locate and select a suitable test provider, the employer can consult commercial directories of recruitment software providers, such as the one available at www.onrec.com or visit trade shows for recruitment testing software providers. When selecting an appropriate supplier, it is good practice for the employer to establish whether or not the test provider has a good track record in developing the type of screening tools that it needs. For example, if the employer needs an online screening tool for a retail assistant role, it could approach other retailers to enquire about the effectiveness of similar tools that they may have commissioned from external providers.

When choosing and developing a system, the employer should take into account:

  • the organisation's recruitment needs, for example the number of candidates that it wants to recruit;
  • the level of integration needed with the organisation's other online systems (for example, an HR database);
  • what functions the system needs to perform;
  • how easy the system is to use; and
  • any additional costs around support and customisation - if the organisation has the IT capability, it may want to consider developing an in-house system rather than procuring one from a third-party provider as this is likely to be a cheaper option for developing and maintaining the system.

Telephone screening interviews

Telephone screening involves the employer conducting brief telephone interviews with applicants prior to the shortlisting process. Screening applicants by telephone is particularly useful if the vacancy is a telephone-based role, for example a call centre adviser, because the interview provides the employer with an opportunity to assess applicants' verbal communication skills.

Typically, telephone screening forms a second screening stage following an initial screening exercise to filter out candidates who do not meet the basic qualifying criteria for the role. It could be too time consuming for the organisation to telephone a large number of people as part of an initial screening exercise.

It is good practice when screening by telephone for the employer to ask the same questions of each candidate (although, if a candidate has already provided the relevant information, the employer can omit a question and record this information in the notes). The employer may also need to ask different follow-up questions of some candidates, depending on their responses. A script setting out what questions the interviewer should ask will help to ensure consistency across candidates.

Telephone interviews at the screening stage can have different structures but usually involve the employer checking whether or not candidates meet certain aspects of the shortlisting criteria. At this early stage of selection, typically telephone interviews do not form an in-depth assessment of applicants' suitability for the post and a 20-minute telephone conversation should be sufficient. However, some employers may wish to carry out a longer and more rigorous telephone screening exercise, for example if the vacancy is for a senior or specialist role.

The employer can use interviewers, for example HR and line managers, to conduct the telephone interview or applicants can be taken through a menu on an automated system. For example, one global retailer used a computer-aided telephone system based on interactive voice-response technology as a screening exercise to fill retail assistant vacancies in one of its new stores. The screening tool required applicants to respond to eight questions over the telephone and around half of the applicants were screened out because they were not available when required or had no retail experience. The retailer had received more than 500 applications for 30 positions, so the screening tool was helpful in reducing the candidate pool to a manageable size.

If the employer uses interviewers to conduct the telephone screening interviews, HR or the recruiting line manager should contact the candidates to arrange a convenient time for the telephone interview and explain its purpose. The interview script should include:

  • the purpose of the interview;
  • the specified questions;
  • an opportunity for the candidate to ask questions; and
  • next steps and timescales.

For example, the telephone screening interview for a customer service assistant role in one call centre takes 20 minutes and the questions include:

  • "What experience have you had working in a customer-facing role in a call centre?"
  • "What experience have you had dealing with multiple phone lines?"
  • "How would you deal with an angry call from a customer?"
  • "How many calls can you deal with in an hour?"
  • "Are you able to maintain accuracy in your typing skills when dealing with a high volume of calls?"

The telephone screening exercise used by one bank for graduate-entry financial services roles is more in-depth and takes up to 35 minutes. Some of the interview questions are competency-based and include:

  • "What skills can you bring to this role?"
  • "Can you give me an example of how you have worked well as a team member?"
  • "What has been your biggest challenge to date?"
  • "Can you tell me about a situation where you have come up with ways to do things differently?"
  • "Can you give me an example of when you have had to make a decision under pressure?"

Some employers may want to use a telephone screening interview to cover other useful areas, for example:

  • the candidate's reasons for leaving his or her current or most recent position;
  • the candidate's reasons for applying for the new role;
  • clarifying any gaps in the candidate's employment history;
  • establishing or confirming recent employment dates; and
  • checking any other details that are unclear and relevant to the application.

The shortlisting process

Following any screening or online sifting, the next stage of the process is for the employer to shortlist candidates to take through to the next stage of the selection process. The panel that decided on the shortlisting criteria should undertake shortlisting with the aid of a shortlisting assessment form.

The shortlisting assessment form

The employer should use a shortlisting form to shortlist candidates for the next stage of the selection process.

A shortlisting assessment form supports a systematic approach to shortlisting by ensuring that the shortlisting panel records relevant evidence to support its decision.

The categories listed on the form should relate to the job-related/shortlisting criteria for the role, for example qualifications, work experience, level of responsibility, competencies and salary level (see Developing shortlisting criteria).

XpertHR provides an example of a form, together with an explanation of the law relating to shortlisting (see Policies and documents > Shortlisting assessment form).

The shortlisting form should include a section for the shortlisting panel to include comments, for example to highlight areas to probe at the next selection stage if the candidate is shortlisted. Each member of the shortlisting panel should complete a shortlisting form for all candidates who have passed the screening or online sifting stage.

Next, the panel should collectively consider all of the forms to decide which candidates to shortlist for the next selection stage. Following the panel's discussion, the recruiting manager should record on a summary form for each candidate, or on a central record, the overall decision on the candidate's application with the reasons why he or she has or has not been shortlisted.

Scoring, ranking and weighting

The shortlisting panel should develop a scoring and weighting system to help it rank candidates in an objective and consistent manner.

When assessing applications, the shortlisting panel should score candidates against the shortlisting criteria. A scoring system with a rating scale enables the panel to allocate a score to each candidate against each category of the criteria. A typical scale for shortlisting is "1 to 5" with "5" indicating that the applicant "exceeds requirements" and "1" indicating that he or she "just meets requirements".

When developing a scoring system, the employer should test it out with a selection of applications to make sure that it is reliable. This means looking at its ease of use for the shortlisting panel, and whether or not the results identify the most appropriate candidates to progress to the next selection stage.

Another useful strategy is for the employer to apply weighting to the shortlisting criteria. This means attaching different levels of importance to certain criteria. For example, the employer could use a simple "1 to 3" weighting framework where "3" indicates "very important", "2" indicates "important" and "1" means "quite important". The employer can assign each shortlisting criterion a weighting, according to its level of relevance to the role.

The following shortlisting matrix demonstrates how weighting can be applied to the shortlisting process:

How weighting can be applied to the shortlisting process
Name Criteria Weighting 1-3 Score 1-5 Total Overall total
Candidate A Degree 2 5 10 19
Experience in customer-facing role 3 3 9
Candidate B Degree 2 3 6 21
Experience in customer-facing role 3 5 15

In the example above, if the shortlisting panel took into account the candidates' scores alone, both candidates would achieve an overall score of 8. However, once the weighting has been applied, one candidate has scored higher than the other because experience in a customer-facing role has been allocated greater importance for the role.

Once candidates have been allocated a total score, the shortlisting panel should rank them in order of their scores. The panel should perform the ranking on a collective basis, once every panel member has individually scored each candidate.

Making a final decision

It is good practice for each member of the shortlisting panel to go through the shortlisting process on his or her own for each candidate. This approach should help prevent panel members being influenced by the views of other people on the panel. When carrying out the shortlisting process, panel members should assess how well each candidate meets the "essential" and "desirable" criteria for the job, and score them accordingly.

Once each member of the panel has decided on his or her shortlist, the panel should compare all of the shortlisting forms for each candidate. It is possible that there will be differences between the panel members' scoring for some or all of the applicants and, if there is, the panel should discuss the reasons for the scoring choices to help clarify in what ways candidates match, or do not match, the shortlisting criteria. The discussion should result in agreement on the final shortlist of candidates.

The number of candidates on the final shortlist will be determined by a number of factors and could be limited by the maximum number of candidates that it is possible for the employer to assess during the subsequent selection process. For example, one selection stage may involve an assessment centre that can accommodate just six candidates. Alternatively, the employer could decide to apply a cut-off point in terms of the minimum score that a candidate needs to achieve for inclusion on the shortlist. However, the employer may need to be flexible and reconsider the cut-off point if too many or too few candidates achieve the minimum score.

If the panel has difficulty identifying a suitable shortlist, it should take into account the following factors:

  • If the final pool of candidates is larger than the shortlist allows: Could the employer increase the size of the shortlist to accommodate additional candidates? This decision is likely to depend on the amount of time needed by assessors for the next stage in the process and their availability.
  • If there is a lack of consensus on whether or not the panel should shortlist some candidates: If the pool of agreed candidates is deemed large enough, any candidates about whom there is disagreement could be placed on a "hold" list. If the employer does not appoint a candidate from the shortlist, it could revisit the "hold" list and include additional candidates on the shortlist to take part in a further selection process.
  • If there are insufficient candidates for the final pool: If the shortlist is too small, the panel should look carefully at the shortlisting criteria and the weighting assigned to them to ensure that it has not excluded candidates with potential too early in the selection process. In some instances, it is acceptable for the employer to agree on a small shortlist as it may still be possible for it to make a successful appointment, for example the post could relate to a specialised role for which the employer would not expect to receive a high number of applications. Alternatively, although small, the shortlist may represent a pool of high-calibre candidates from which the employer is able to make a successful appointment.

Avoiding bias and discrimination in shortlisting

Using appropriate job-related (or shortlisting) criteria is essential to avoid bias and discrimination when shortlisting (see Developing shortlisting criteria). Without criteria, the organisation may end up with an unsuitable pool of candidates. Further, potential claims of discrimination will be harder to defend.

One way of ensuring that only relevant information is taken into account when shortlisting is for the employer to remove all personal information about candidates before it gives CVs and application forms to the shortlisting panel. Name, sex, age and marital status may consciously or unconsciously influence some individuals on a shortlisting panel, even though these personal details are not relevant to the application. For example, an individual on the shortlisting panel may reject a candidate because he or she perceives the candidate to be too old or too young for the role.

Personal details that are required for the processing of the application should be contained on a tear-off portion of the form rather than being integrated into the main application form. The tear-off portion can then be removed from the main part of the application form, before the application is passed to the relevant line manager to assess. In this way, the line manager cannot be influenced by the candidate's gender, marital status, race or age and so the risk of discrimination is minimised. The HR department will still have access to personal information that it may need during the recruitment process.

A CV and covering letter may contain personal information, for example marital status and age, that it will be difficult to hold back from the shortlisting panel. If this is the case, the employer could blank out this type of information before it gives the documents to the shortlisting panel. Using a cross-reference on the CV or application form, and keeping a record of which number relates to which candidate, means that names are not needed until the later stages of selection.

One potential challenge relating to discrimination is length of experience. Traditionally, recruiters have asked for a minimum number of years' experience, but this approach can lead to age discrimination, for example if a younger job applicant has not had the opportunity to accrue a specific number of years in a role. Further, shortlisting candidates on the basis of a certain number of years' experience could mean that the employer discriminates against women who have taken time out of the workplace to raise children or applicants who have needed time away from work due to a disability. The employer should think about what a candidate with the relevant experience should be able to do and define the job requirements in those terms when it develops the shortlisting criteria. For example, the post may need an individual who has experience of handling several long-term projects. Having a certain number of years' experience says nothing about the quality of the experience gained by the individual. For shortlisting, the panel should ensure that it focuses on how a candidate describes his or her responsibilities, achievements and tasks rather than on how many years' experience he or she has.

Employers that use social media to assess the suitability of potential new recruits need to take care not to discriminate unlawfully. Often, an individual's social media page will contain personal information relating to characteristics such as race, religion, sexual orientation and age. These characteristics are protected under the Equality Act 2010. Where an employer views such information about a job candidate, this may influence (consciously or unconsciously) the employer's decision. If the employer rejects the candidate (whether or not the information on social media has a bearing on this decision) the individual might bring a claim of unlawful discrimination to an employment tribunal arguing that the real reason for the rejection was the protected characteristic.

The data protection requirements that apply to personal data that an employer obtains and processes during the course of a recruitment exercise apply also to personal data obtained from social media sites.

In essence, employers should use social media to investigate candidates only if there is a sound reason for doing so and one that is relevant to the recruitment decision (for example, social media use may be relevant to the job). Employers should also inform job applicants if they propose to look at their social media page as part of the recruitment process and allow them to respond to their interpretation of the content.

A key strategy to help avoid bias and discrimination is for the employer to avoid making assumptions about candidates. The employer should provide training in equal opportunities and diversity for all managers responsible for recruitment so that they are aware of the risk of making prejudiced decisions at the shortlisting stage. The shortlisting panel should consider the evidence supplied by a candidate to demonstrate how he or she meets the shortlisting criteria, and avoid making assumptions that are not based on factual evidence. The following are examples of pitfalls that employers should avoid:

  • The panel should not assume that an individual with a visual impairment cannot do a job that requires a lot of reading. There are technologies and support available for the employer to make reasonable adjustments (see Equality and diversity > Disability).
  • The panel should not assume that a woman cannot do a job that requires a lot of lifting. This is the type of job requirement that, if necessary, the employer can confirm at the assessment stage with all candidates, including male candidates.

Similarly, the shortlisting panel should take into consideration the presentation of a CV or application form only for certain vacancies, for example for a design role or if the job requires good written communication skills. If this is the case, the employer should include this competency as a criterion for the job. Recruiting organisations should be aware that a third party may have written the CV or application form on the candidate's behalf, so the employer cannot assume that the application is a good reflection of the candidate's communication and presentation skills. In some cases, recruitment agencies rewrite or reformat CVs to their house style.

Case study

A medium-sized IT consultancy firm decided to hire two trainees to work on its customer helpdesk. The HR manager agreed with the helpdesk supervisor that the shortlisting panel should include:

  • the HR manager;
  • the helpdesk supervisor; and
  • a member of the customer liaison team responsible for looking after customers (as the consultancy was working on improving its customer service, the HR manager wanted an individual with a good understanding of customer concerns and needs to serve on the panel).

The HR manager convened an initial meeting of the panel to discuss and agree the shortlisting criteria, based on the job description and person specification. The panel ensured that the criteria were all qualities that could be assessed from the application form that candidates would need to complete. The panel identified the following shortlisting criteria that candidates would need to satisfy:

  • Qualifications: GCSE (or equivalent) grade "C" or above in an IT subject.
  • Experience: Experience of working in a customer-orientated role, which could include voluntary work.
  • Knowledge and skills: Working knowledge of Microsoft Office applications and proficiency in using databases to capture and analyse data.
  • Behaviour: Evidence of situations where the candidate had dealt with customers in a calm manner when under pressure.
  • Motivation: An interest in a career in IT.
  • Practical requirements: A willingness to work shifts between the hours of 7am and 7pm, Monday to Friday.

The HR manager prepared an application form that covered questions relating to the shortlisting criteria. For example, one question on the form asked candidates to "Give an example of when you have dealt with a difficult customer". The front sheet, for candidates to provide their name and contact details, was designed to be detachable. Following receipt of the application forms, the HR manager ensured that an independent third party who was not involved in the recruitment exercise removed the front sheet from each form. This helped to reduce the risk of the shortlisting panel making biased decisions based on candidates' personal circumstances or characteristics.

The HR manager created a shortlisting assessment form with a scoring system of 1 (unacceptable), 2 (satisfactory) and 3 (good). The HR manager weighted the shortlisting criteria to help ensure that the shortlisting process was effective and applied an in-depth assessment of candidates' suitability based on the most important criteria for the role. The shortlisting panel had agreed not to limit the number of suitable candidates to be shortlisted. It set a minimum total score of 20 that a candidate would need to achieve across 10 criteria, with the need for a minimum score of 2 on each of the shortlisting criteria.

More than 50 applications were received, so the HR manager carried out an initial screening exercise using basic qualifying criteria, for example by filtering out all candidates who did not hold a GCSE grade "C" qualification or above in an IT subject or did not have experience of working in a customer-oriented role. Through this process, the manager screened out 30 applications.

The HR manager gave a copy of the remaining 20 applications to the other panel members, together with a copy of the shortlisting assessment form. After a week, the panel members met to discuss their decisions. First, the panel identified the candidates whom they all agreed were either a "yes" or a "no", and allocated six applications to the "yes" file and six to the "reject" file. Second, the panel members considered the remaining eight applications and discussed the breakdown of the scores and the reasons for their shortlisting decisions. Out of these eight applications, the panel assigned four to either the "yes" or "reject" file and decided to keep the four applications on which they could not agree in a "hold" file to revisit if the organisation needed to expand the shortlist in the future. Finally, the HR manager recorded the panel's decision on each candidate and, if unsuccessful, the panel's reasons for the decision. The shortlisting process resulted in a final shortlist of eight candidates to progress to the selection process.