What Is Adverse Impact? Definition, Testing & How-To Reduce 

Humans are (in many ways) hardwired for bias—it can be a major challenge in recruiting, social interactions, and everyday life. We all see the world through different lenses, and those lenses lead to real consequences. In many cases, these are harmless. Maybe you prefer coffee to tea or think cats are better than dogs… no big deal.

But, when biases are left unchecked in recruitment, they can have negative consequences. 

Adverse impact is the concept used to describe these negative consequences. It occurs when employment practices disproportionately affect protected groups—and it’s a big problem in the workplace.

So, if you’re involved in the hiring process, it’s important to understand what adverse impact is and how you can reduce it. In this Willo guide, we’ll explore the concept of adverse impact, its impact on diversity and inclusion efforts, and what you can do to combat it.

But first…

What Is Adverse Impact?

Adverse impact is the negative consequence someone faces because of a different person’s bias—in short, it’s what happens when bias turns into discrimination (inadvertent or not).

For example, imagine a company is hiring for a role. 

They receive 200 applications with roughly equal qualifications—95 from men, 100 from women, and 5 from non-binary individuals. If 90% of those selected for interviews are men, this is evidence of adverse impact—women and non-binary individuals seem to be disproportionately excluded from the interview process.

We say evidence here because adverse impact is provable. United States law, for example, looks at adverse impact as a statistical concept. But there are specific math calculations (which we’ll cover in a moment) that you can use to determine if it’s present.

How to Test for Adverse Impact 

Adverse impact can be subtle or hidden—here are three of the most common methods for uncovering it:

1. Four-fifths rule (80% rule)

This simple, easy-to-use rule of thumb for detecting adverse impact compares the selection rate (the percentage of people hired or promoted) for a protected group to the selection rate for the majority group. It’s also the rule the US Equal Employment Opportunity Commission (EEOC) uses to evaluate discrimination claims.

The rule works like this: if the selection rate for a protected group is less than 80% of the selection rate for the majority group, there may be adverse impact at play.

Here's how it works:

1. Calculate the selection rate for each group. This is the number of people hired or promoted from each group divided by the total number of applicants from that group. For example, say you’re comparing selection rates for non-disabled people (the majority group) and disabled people (the minority group). If 80 out of 100 non-disabled applicants are hired, the selection rate would be 80%. If 5 out of 20 disabled applicants are hired, the selection rate would be 25%.
2. Apply the 80% rule. Next, calculate 80% of the selection rate for the majority group. In this case, 80% of 80% is 64%. This means that the EEOC would expect the selection rate for disabled people to be at least 64%. In our example, the selection rate for disabled people is far lower—25%, as calculated earlier. This is strong evidence of adverse impact.

2. Chi-square test

In reality, there are multiple non-discriminatory reasons why two groups might have different selection rates. A Chi-Square test compares the actual number of people selected from each group to the number you’d expect to be selected if selection rates were consistent across all groups.

 To illustrate, let’s imagine the same hiring scenario as before:

• 120 applicants total (100 non-disabled, 20 disabled)
• 85 hires total (80 non-disabled, 5 disabled)

Here’s how the test works:

1. Calculate the overall selection rate. This is the percentage of people that were selected (for hiring, promotion, layoffs, etc.). You calculate this by dividing the total number of selected people by the total number of people and multiplying by 100. 

In our example, the selection rate would be 85 (total hires) /120 (total applicants) x 100 = 70.8%.

2. Calculate the expected number of selections for each group. This is done by multiplying the total number of people in each group by the selection rate calculated in step one. 

In our example, the expected number of hires for non-disabled applicants would be 100 x 70.8% = 70.8 (rounded to 71). The expected number of hires for disabled applicants would be 20 x 70.8% = 14.16 (rounded to 14).

3. Compare the expected and actual number of selections. This is easiest if you create a table with the expected and actual numbers side by side. Big differences between the two numbers may indicate potential discrimination (either positive or negative).

Expected Number of Hires

Actual Number of Hires

Difference

Non-Disabled Hires

71

80

+9

Disabled Hires

14

5

-9

4. Calculate the p-value. Simply looking at the chart will give you an indication if bias is present. In this example, disabled people were severely underrepresented on a proportional basis. But if you want a statistical measure, calculate the p-value, which tells you whether the differences you’re seeing between expected and actual numbers could have just occurred by chance or if there is a difference that warrants further investigation. To calculate it, we recommend using an online calculator (unless you have a stats background). 

For our example, we input the expected and actual numbers for both non-disabled and disabled hires, and the calculator tells us that the difference is statistically significant—meaning adverse impact is likely present.

Keep in mind that the Chi-Square test only performs well for large data sets. So, if you’re working with a smaller sample size, the next test is more suitable. 

3. Fisher's exact test

Fisher’s Exact Test is an alternative to the Chi-Square test that works better for small sample sizes. It calculates the probability that your data could have happened by chance, assuming there is no real difference between the two groups. A very low probability that your data happened by chance (usually less than 5%) suggests potential adverse impact.

Social Science Statistics has a great calculator for Fisher’s Exact test, so you don’t need to do the calculations manually. Just input your data, and it will give you a p-value. That p-value will tell you if there is a statistically significant difference between the two groups.

All three methods have their strengths and weaknesses. The four-fifths rule is the easiest to understand but least precise. The Chi-Square test is more powerful but requires larger sample sizes. The Fisher's Exact Test is a good compromise for smaller datasets.

Consequences of Not Paying Attention to Adverse Impact

Legal ramifications

Adverse impact can violate anti-discrimination laws in many countries. For example, multiple countries, including the United States, Canada, the UK, and the EU, have laws prohibiting discrimination in hiring and employment practices. If your selection practices are found to be disproportionately impacting protected groups, you could face legal action. 

Ethical concerns

Everyone wants to feel like they have an equal opportunity to succeed, regardless of background. When you ignore adverse impact, you intentionally create a system that disadvantages certain groups of people. 

This raises huge ethical questions about fairness and equality, particularly if any part of your corporate mission is about acting with ethics or integrity. Discrimination has no place in the workplace, and as a company, it is your responsibility to ensure that your hiring processes are fair and unbiased.

Reputational damage

Today's talent pool is more diverse than ever before. People want to work for companies that value diversity, equity, and inclusion (DEI). In fact, studies show that 80% of employees consider DEI a factor when choosing a job. 

If your company is perceived as having biased practices, it can damage your employer brand. And as we found in our 2024 Hiring Trends Report, more than 83% of recruiters see a strong employer brand as either “Very…” or “Extremely important”.

Source: Willo 2024 Hiring Trends Report

Decreased employee morale

When employees from certain groups notice they're being passed over for promotions or opportunities unfairly, it can have a significant impact on morale. They may feel discouraged, disengaged, and less likely to contribute their full potential. This can lead to a decrease in productivity, creativity, and, in the worst cases, employee turnover.

Missed opportunities

Diverse teams bring a wider range of perspectives and experiences to the table, which can lead to better problem-solving, decision-making, and up to 19% better innovation. By ignoring adverse impact, you're potentially missing out on this large pool of qualified talent and the benefits they bring. 

How to Reduce Adverse Impact 

1. Structured employment interviews

A structured interview is exactly what it sounds like—an interview process with clear guidelines that prevent recruiters and hiring managers from relying on gut instinct alone.

At the most basic level, this means using the same set of questions for every candidate. But, for the best results, you can also standardize almost every aspect of interviews and screens, including:

• Time limits
• Question order
• Review and evaluation criteria
• Interviewer disposition

How can you implement all this? By switching to async video interviews. Here at Willo, we offer an asynchronous screening solution that helps standardize interviews and screens while also reducing the potential for bias. 

You can use our intelligent question generator to create questions for the interviews. The tool only gives you questions that are highly relevant to the role. As a result, you can keep any interview question-induced adverse impact to a minimum. 

After deciding on your questions, use them to set up and publish an interview directly on Willo. We also allow you to decide on duration, number of retakes, and time to think. It’s so easy that you can finish the entire setup within minutes. 

After publishing your interview, we provide you with invite links to send to interviewees. They can respond to the questions via video, text, or audio on their own time without any pressure from you or anyone else.

2. Create interview scorecards

Structured interviews are a great first step, but they can be even more effective when paired with interview scorecards.

An interview scorecard lists the key qualities you’re seeking in a candidate, like communication, willingness to learn, and enthusiasm. Then, each quality would have defined rating scales (e.g., "Strong no,” "No," "Maybe," “Yes,” “Strong Yes”), and interviewers would rate each candidate on these scales. 

This allows you to keep your evaluation process transparent, leaving no room for bias.

We can lend you a hand here, too…

After your candidates submit their responses on Willo, invite your hiring team to collaborate and create an assessment scorecard for responses. 

Then, you can simply rank and compare the Willo scorecards to pick your best candidates.

3. Focus on core competencies

When evaluating candidates, it's easy to get caught up in specific requirements or past experiences. However, the key is to focus on the core skills and abilities essential for success in the role. 

Here’s what we mean. Instead of asking for "5 years of sales experience," rephrase the requirement as: "Proven ability to build rapport with new clients, identify customer needs, and achieve sales quotas through effective communication and presentation skills." 

Here's how to make this work: 

• Job analysis: Identify the KSAs (knowledge, skills, and abilities) needed to excel in the role. You can spot these by thinking about how the job supports the overall business. If possible, talk to current employees in the role as well to get firsthand perspectives. 
• Skills-based descriptions: Instead of listing years of experience as the primary requirement, focus on the skills and knowledge needed to perform the job. For example, instead of "5 years of experience in marketing automation," list the specific software proficiency and campaign management skills you need.
• Behavioral interviewing: Ask candidates situational questions that explain their past behaviors in similar situations. This gives them an opportunity to share any lessons learned or how they might act differently today based on new knowledge.

One example of this is SmartLynx Airlines. Using Willo, the international airline built a human-centric employer brand and focused on holistic candidate assessment to understand the whole candidate, not just their resume or application details. This helped them source and hire 400+ people from around the world.

4. Share best practices across recruitment stakeholders

Bias can creep in at any stage of the recruitment process—from sourcing candidates to making final decisions. To combat this, foster collaboration and knowledge sharing among everyone involved in hiring. 

Here are some ways to achieve this:

• Regular training sessions: Conduct training sessions for recruiters, hiring managers, and interviewers on unconscious bias and best practices for inclusive hiring. There are plenty of high-impact training methods for fostering self-awareness, like immersive workshops, VR experiences, and personal bias testing.
• Create SOPs: Develop SOPs that systematize inclusive hiring. You can create these for tasks like writing job descriptions, sourcing candidates, conducting interviews, and making offers. 
• Support open communication: Encourage open communication between different stakeholders in the recruitment process. Recruiters can share their strategies for sourcing diverse candidates, while interviewers can provide feedback on the effectiveness of the interview process.

5. Create a clear and equitable promotion policy

Just like hiring, promotions are also susceptible to adverse impact. To prevent this, establish a clear and well-defined promotion policy based on objective criteria:

• Performance metrics: Develop a transparent system for evaluating employee performance that aligns with the company's goals and the specific requirements of each position. This could include metrics like sales targets achieved, project completion rates, or leadership skills demonstrated.
• Succession planning: Implement a structured succession planning process that identifies high-potential employees and provides them with opportunities for development through mentorship, training programs, or challenging assignments.
• Diversity in promotion committees: Promote diversity within promotion committees. By including individuals from different backgrounds, you can ensure a wider range of perspectives are considered during the evaluation process.

Build a Fairer Workforce With Willo 

Remember—adverse impact can be subtle, but its consequences (missed opportunities, reinforced biases, and social stereotypes) are often devastating. By adopting structured employment interviews, interview scorecards, and the other strategies outlined here, you can significantly reduce this unintentional bias in your selection process.

As you can see, Willo is a valuable tool in your journey toward a more inclusive workplace. Our platform allows you to easily create structured interviews and introduce transparency in your screening process with Scorecards. 

Ready to get started? Sign up for your free Willo trial today.