Can AI bridge the confidence gap for women in senior positions?

It's well known that men tend to overestimate their abilities while women under-sell themselves when applying for roles - just one of many cultural factors influencing the gender pay gap. Could artificial intelligence help overcome this challenge? Alex Cresswell examines how AI could help.

The recent publication of gender pay gap information has shone a very clear light on the national shortage of women in senior positions in UK companies.

HSBC, for instance, has highlighted the fact that only 23% of its senior positions are held by women and has cited this as a significant driver of its gender pay gap of 59%.

This has spurred many organisations to consider how they can address the imbalance and a number (including HSBC) have declared their support for goals such as those proposed by the 30% Club, which is campaigning for women to occupy at least 30% of senior roles by 2020.

This is a positive step forward; it is not so long ago that the issue was not even acknowledged. Now that we have identified targets, the conversation can move on to how we can actually achieve change.

The overconfidence gap

In order to accomplish lasting change, we need to address a host of issues ranging from cultural challenges and a lack of role models to the more subtle gender biases at play that result in women receiving less frequent and less robust feedback than many of their male counterparts.

One well reported issue is the confidence (or overconfidence) gap, as highlighted by psychologists David Dunning and Joyce Ehrlinger in 2003.

This piece of research came up with the oft-quoted finding that men tend to overestimate their abilities and performance whilst women tend to underestimate theirs. This presents a particular challenge in terms of encouraging people to apply for the more senior roles.

It is a pernicious problem - how can we increase the number of women in senior roles if they are already outnumbered by men and are less likely than men to apply?

Companies striving for gender balance often require a balanced shortlist of candidates, but this can introduce a delay in the process and that time lag can have a serious business impact which then erodes the support for gender balance initiatives in the hiring manager community.

Or, even worse, we end up with candidates on the list who are there simply to make up the numbers - something which serves the business and the candidates very poorly.

Potential to succeed

The key then, is to be able to proactively identify those individuals who have the potential to succeed in a new role and then encourage them to apply.

This presents two challenges - how to find them and how to address the hesitation to 'go for it'. Technology and artificial intelligence can help us with both.

Firstly, it can help us move towards a more proactive approach to recruitment and career mobility through providing an additional piece of objective and external verification that individuals demonstrate a clear 'fit' to a role.

This may well nudge people towards seeing greater alignment with a new opportunity than they might have otherwise.

Secondly, technology, and in particular AI, can also help us find 'hidden' talent through revealing some of the less obvious factors and commonalities that lead to success.

To understand this additional lens AI can offer, we should begin by taking a look at how we select people for roles today.

Objective assessment

Traditionally there has been a dependence on competency frameworks to define what good looks like in a role and to underpin the assessment processes. This has been the industry standard for decades.

Much work has been undertaken to develop best practice and improve the objectivity of their use, including careful training of assessors through to the creation of carefully structured assessment marking frames.

But despite this effort, subjectivity and inconsistency can still creep in and we are still limited by the amount of information that we, as humans, can observe and manage at any one time.

The advent of technology such as artificial intelligence, however, provides an opportunity to combine other types of insight to enhance the assessment picture.

Gender balance at Unilever

Unilever for instance, has been using tools from AI recruitment tech provider Pymetrics to automatically identify which candidates they should be interviewing.

This technology uses online neuroscience games to gather data on behavioural traits of people - essentially their cognitive and personality makeup - and then deploys AI to look for patterns amongst this huge set of data.

This means that it is possible to create a detailed, objective model that is free of unconscious bias and that accurately describes and predicts what it takes to be successful through looking at what the data tells us is common and distinctive about high performers.

When used for selection, as it was for Unilever, it has helped them to maintain gender balance in their graduate intake (they are one of the few UK companies that had a gender pay gap in favour of women, as they have more men in their lower paid manufacturing roles).

When used for sourcing, as it was for a global financial institution, it has helped them change their hiring pipeline for some of their analyst roles from an 80/20 split in favour of men to 50/50 for the last two years.

Greater insight

Consider then how this technology - and others like it - can help us to address the confidence gap inside our organisations, and the range of other barriers in the way of career progression, by providing a rich additional source of insight.

It enables us, at scale, to identify suitable candidates and employees who have the potential to succeed in a more senior position and then we can invest in their development knowing that the chances of success are high.

In this way we can rapidly accelerate our ability to provide hiring managers with a balanced slate of candidates and provide evidence for why they should be considering people who may have less experience on paper, thereby helping to take some of the risk out of moving towards greater diversity.

The really important word here is scale. This is what technology is good at - taking a process and making it ubiquitous and easy.

By harnessing this opportunity, we can quickly make some of today's hiring challenges look antiquated and, hopefully, move rapidly on from the 30% goal. Because why do we have a goal of only 30%, when women make up half the population?