Our Expert Guide to Creating and Using Candidate Interview Scorecards

Hiring decisions are important. And like any important decision, they require careful consideration and evaluation, not gut reactions and instincts.

According to Aptitude Research’s 2020 Talent Acquisition study, 63% of companies reported identifying quality hires as their greatest challenge. And the leading reason? A lack of consistency and objectivity in the evaluation process.

LeadershipIQ research found that 62% of HR executives feel their hiring managers lack consistency in interviewing candidates. This can lead to poor hires, high turnover rates, and major costs for businesses—the cost of replacing an employee can be up to 1.5X to 2X their salary. To avoid this, you need to make better, more data-driven hiring decisions using candidate scoring systems that prioritize key skills and attributes.

In Willo’s State of Hiring 2024 Report, we found that 21.9% of recruiters rated skills shortages as a key hiring challenge. A better system for evaluating candidates—like interview scorecards—can help address this issue by focusing on the specific skills and qualities needed for each role.

So, stick with Willo as we walk you through the process of creating interview scorecards that help you identify top talent more reliably.

What Are Interview Scorecards?

Interview scorecards are a recruiting tool designed to standardize and quantify candidate evaluation. Depending on the company, they go by quite a few names (scoring sheets, rating guides, candidate scorecards), but they all serve the same purpose—to streamline and improve the hiring process. 

How do they work?

An interview scorecard is a grading rubric that outlines the key skills, qualifications, and qualities you’re looking for in a candidate. During or after an interview, interviewers score candidates for each criterion on the scorecard.

Once all interviews are completed, the scores from each interviewer's scorecard can be aggregated to create an overall assessment of the candidate. This can then be compared against other candidates to determine who might be the best fit for the job based on the round of interviews.

At Willo, we’ve incorporated a Scorecards feature into our asynchronous video interview solution. This customizable digital rubric enables every stakeholder in the recruiting process to rank candidates based on pre-defined criteria.

‍

Here are a few key things to know about our interview scorecards:

• You can customize Willo Scorecards, defining each criterion and adding a description using your organization’s language. 
• You can create multiple Scorecards in your Willo dashboard. Then, assign a Scorecard to a given job opening or interview process to ensure you’re assessing candidates on appropriate criteria. 
• You can review candidate rankings as a group or compare one-on-one in the Scorecards Dashboard.

With Scorecards, you don’t just have to rely on random impressions or gut feelings. Instead, you have concrete metrics and valuable analyses to help guide your hiring decisions in the best possible direction. 

The Benefits of Interview Scorecards

Interview scorecards can be vital in your interview process, offering a structured approach to evaluating candidates. 

The major benefits of using interview scorecards include:

Hire confidently

This first benefit is a simple one—using interview scorecards allows you to feel more confident in your hiring decisions. 

With a clear set of criteria and measurable metrics, you can trust that you are making the best choice. Plus, interview scorecards allow you to compare candidates objectively, rather than relying on subjective impressions.

Stay consistent across interviews

In any interview process, but especially in high-volume hiring, you want to ensure that your evaluation is consistent. Interview scorecards can ensure that your candidates go through a more fair interview process by using the same criteria for all candidates.

Let’s not forget that this is quite important from an diversity, equity, and inclusion (DE&I) standpoint. By using consistent, job-relevant interview scorecards, recruiters can help minimize potential discrimination or bias in the hiring process and support diversity and inclusion.

Reduce bias

One of the most significant benefits of interview scorecards is their role in reducing unconscious bias in the hiring process. By focusing the interviewer’s attention on specific, job-related competencies and predefined questions, scorecards help prevent biases based on gender, race, age, or other unrelated factors from influencing the final hiring decision.

Improve interview focus and efficiency

Interview scorecards can help guide interviewers to focus on the most critical aspects of the job and the candidate’s fit for the role. This structured approach can lead to more efficient two-way interviews down the line as it keeps conversations centered on evaluating key skills and competencies. 

Facilitate collaborative hiring decisions

Interview scorecards facilitate collaborative decision-making by providing a quantifiable overview of each candidate’s strengths and weaknesses. They allow hiring teams to discuss candidates’ qualifications based on shared data and different perspectives on job fit, leading to more informed and consensus-driven hiring decisions. 

Provide better feedback for applicants

By taking notes and scores during your interview process, you can give valuable feedback to unsuccessful candidates. This feedback will give them a suitable direction for their employment journey, and you’ll be strengthening your employer brand simultaneously. 

Valuable documentation

Scorecards can be quite useful as records of your interview process. You may need them for future reference, audit purposes, candidate feedback, and post-hiring analysis to vet the efficiency of your hiring process. 

You might also not hire a candidate for a specific role. However, having their video interviews and scorecards in your ATS could help you find a great fit for a different role during the same hiring round or in the future. This can be invaluable for building a talent community you can pull from. If using this approach, keep candidates engaged genuinely; don’t just decline them now, ignore them, and reach back out later. 

Potential Challenges of Interview Scorecards

While interview scorecards offer many benefits, there can be potential challenges in their implementation, such as ensuring they do not restrict the natural flow of interviews, or limit the depth of conversation with or responses of candidates. So, proper training and design of your scorecards can mitigate these issues. 

Here are a few potential challenges to look out for:

• Limits on spontaneity and flexibility: Scorecards may restrict the natural flow of conversation during live interviews. That said, we don’t believe flexibility is required (or even beneficial) in interviewing. Willo’s one-way interviews are so effective because every candidate gets the same questions, order, and amount of time—just ask airBaltic or Endava. Our Scorecards complement this structured approach by giving recruiters a clear set of criteria to evaluate each candidate on.

Source: How airBaltic screened over 100 high-potential candidates a week with Willo

• Lengthy prep and implementation time: Developing and implementing effective interview scorecards from scratch requires significant upfront effort. This includes defining criteria, training interviewers on how to use scorecards properly, and ensuring consistency across the interviewing team. Again, Willo Scorecards eliminates this challenge by providing a user-friendly template that can be easily customized and shared in minutes.
• Potential overemphasis on quantitative evaluation: Design your scorecard with multiple outcomes in mind. For example, if your scorecard doesn’t account for problem-solving skills, communication styles, or emotional intelligence, such criteria won’t count toward the final score. 
• Risk of misinterpretation and misuse: The effectiveness of your interview scorecards heavily depends on the consistency of their use by hiring team members. Misinterpretation of questions or criteria, as well as inconsistent scoring, can lead to unreliable assessments and undermine the objectivity scorecards are designed to ensure. To address this, design your scoring criteria with job outcomes in mind (more on that in the next section!)
• May not fully eliminate bias: While scorecards are designed to reduce hiring biases, their effectiveness in removing them entirely is limited by the biases that could be present in the criteria selection and question formulation. So, if the scorecard itself is not carefully designed to be inclusive and unbiased, it may perpetuate existing biases within your hiring process.

Best Practices for Creating and Using Interview Scorecards

Define your evaluation criteria clearly

Before you implement your interviews, it’s important to establish clear, job-specific criteria that you can use to evaluate your candidates. The criteria should relate to the role’s requirements and include hard and soft skills.

For instance, while a candidate may be personable, empathetic, and communicative, the role you are looking to hire may need someone who is data-driven and analytical. By defining your criteria beforehand, you can ensure that interviewers assess the candidates based on the same standards.

Use a consistent rating system

Adopt a uniform rating system across all scorecards to ensure consistency in evaluating your candidates. We advise choosing descriptive labels, such as ‘poor’ to ‘excellent,’ to reduce misunderstandings. 

Consistency in your rating system helps to compare candidates’ performances accurately and make more informed decisions as a team. 

Review and discuss scorecards collaboratively

After your interview rounds, gather the hiring team to review and discuss the scorecards together. This collaborative review allows for a comprehensive evaluation of each candidate from multiple perspectives. You’ll be able to highlight discrepancies in scoring and incorporate all stakeholder feedback to bring greater value to your decisions. 

Regularly update scorecards for continuous improvement

Interview scorecards should not be static. By regularly reviewing them—and changing them depending on the job you’re recruiting for—you can update any criteria based on emerging needs, hiring outcomes, feedback from your team, and organizational priorities. Ultimately, this helps ensure your recruitment process stays relevant and effective in the face of change. 

Train your hiring team 

While scorecards may seem straightforward, you want to ensure everyone on the hiring team knows how to use them effectively to get reliable and consistent results. This includes understanding the evaluation criteria, applying the rating system, and documenting extra observations. Proper training minimizes the risk of misinterpretation and inconsistency in scoring.

How Are Willo Scorecards Different From Other Interview Scorecards?

Much thought and research went into the design of Willo Scorecards—but how exactly are they different from your average interview scorecard template? 

The most important differentiator is the rating methodology. 

Many other platforms with rubrics or scorecard features use a Likert scale—a general ranking of 1-5 or 1-10. In our conversations with recruiters, we found that this model leaves room for bias and misunderstanding. 

For instance, a ranking of “5 out of 5” might be defined as a “very good candidate.” However, every reviewer will have a slightly different understanding of “good,” leading to inconsistent rankings. It’s also possible that some individuals have a personal belief that any perfect score (e.g. 5 out of 5) is simply impossible, so they won’t give any candidates that ranking even if the description warrants it. Further, Likert scales are less effective with small sample sizes—if you only have 2-5 people reviewing each candidate, the scores could wildly vary.

The Willo Scorecard methodology is based on recruiter needs and human psychology research. 

Instead of a 1-5 or 1-10 ranking, you’ll see the following: 

• Strong Yes — Endorse this hire: This indicates the candidate fulfills the required qualifications and brings additional strengths that could greatly benefit the role. 
• Yes — Consider for hire: This indicates that the candidate has the necessary skills and qualifications to perform the job effectively, and should be considered a viable hiring option. 
• Maybe — Need more information: This indicates that the candidate shows promise, but there are areas that require further investigation before making the final decision. Additional interviews, skill evaluations, or reference checks are recommended to understand their fit fully. 
• No — Should not consider hiring: This indicates that the candidate does not meet the essential requirements of the position, or there are specific concerns that the review feels preclude them from being a good fit. 
• Strong No — Oppose this hire: This indicates a strong recommendation against hiring the candidate. The assessment is based on significant gaps in the candidate’s qualifications or identified issues that strongly suggest they are unsuitable for the role or the company’s culture. 

Each outcome is tangible and related to the outcome—a confident hiring decision.

We also ensured that a “Maybe” left no room for fence-sitting—we’re all human, and it’s fair to meet a candidate and like them but realize you need to know more about them. This ranking allows for that reality and prompts the conversation of what information you need to decide.

You’ll also notice different colors on an easy-to-recognize scale, making rankings even more consistent. 

4 Ways to Use Willo’s Interview Scorecards

Scorecards are powerful tools used in various stages of the recruitment process. 

Here are four ways we’ve seen recruiters use Willo Scorecards to make more confident hiring decisions:

1. Holistic candidate screening

Your hiring process likely has more in-depth interviews to assess role fit, team fit, culture fit, and skills fit. However, you can use your screening step to test the very basics of every piece. 

How to make this work:

• Ask candidates to share examples or previous experiences. 
• Watch or read their responses. 
• Make sure there are ranking criteria for each category on your scorecard.
• Let stakeholders rank candidates based on whether they provided enough information to justify further interviewing.

2. Key skills assessment

If your role requires multiple foundational skills, without which a candidate cannot do the job, build an interview scorecard that identifies each need. 

How to make this work:

• Ask for experiences or times when someone demonstrated the key skills.
• Pay attention to softer skills in how someone answers questions (e.g. are they outgoing and friendly in their tone? Or more reserved?).
• In your Scorecard, explicitly call out both needs and let stakeholders rank candidates. 

3. Making data-driven go or no-go decisions

After all the candidates have been ranked, you’ll see every score in the Scorecards Dashboard. Use this tool to make quick, data-driven decisions about which candidates to move forward with versus reject.  

For example, the Ranking view makes it easy to automatically move forward all highest-scoring candidates and reject all lowest-scoring candidates. This is similar to a Likert scale, but you can make the decision with more confidence because a top-ranked candidate is one that stakeholders would advocate for, whereas bottom-ranked candidates are those that stakeholders explicitly would advocate against. 

You can make quick and informed decisions because you have this tangible definition from stakeholders. 

If you have candidates ranked as “should hire, but not necessarily going to advocate for them,” you can have an internal conversation about whether you need more candidates to move through, whether to put them on a waitlist, or whether to reject them if you have enough top-scoring candidates in the pipeline. 

If you have any candidates in a “needs more information” ranking, you can contact them to collect what you need while concurrently moving forward with your strongest candidates. This helps ensure no amazing candidates fall through the cracks without slowing down your overall process. 

4. Individual candidate comparison

Sometimes, your decisions come down to two candidates, each with close scores, but you can only pick one to proceed with. In these cases, use the Scorecards Dashboard to compare candidates one-on-one. 

You will be able to see holistic scores for candidates and the scores each stakeholder provides. Using this data, you can quickly follow up to gather any additional information from stakeholders or the candidate, giving you what you need to decide.

Make more confident hiring decisions with Willo

Hiring someone is expensive—hiring the wrong person is even more expensive. With money and the future of your organization (and your customers) on the line, each hire matters.

When you use Willo, you’re saving admin time and collecting high-quality data about a candidate in the form of video or audio interviews, text answers, and document uploads. Score them objectively, fairly, and consistently with our Scorecards, then analyze the results in aggregate or 1:1 so you can make a hiring decision with the highest possible confidence. 

Sign up today to start making more confident hiring decisions.

‍