Connecting the Dots Blog

A Puzzling Scenario – Part II

An Even More Puzzling Employee Selection Scenario

 In our prior blog, “A Puzzling Scenario – Part 1,” we explored a hypothetical employee selection situation. A selection test with “perfect” validity was used for employee selection without the understanding of the important role that selection ratios play in a sound, rigorous employee selection system.

Today, we’re going to explore the impact of the opposite case; that is the use of a selection test with zero validity or no correlation to job performance.

 

Selection Ratios Defined

A selection ratio is the number of applicants screened (e.g. five) compared to the number of applicants selected (e.g. one).

 Assuming that a hiring manager is using a valid performance predictor, the higher the selection ratio, the more likely that hiring manager is to find a candidate with the potential learn and perform well on a job.

An All Too Common Occurrence

 We have seen instances in which hiring managers or HR personnel learn about a test, assume it’s valid, and decide to use it in their employee selection process. In our experience, such tests often are instruments like basic literacy tests or a personality style assessment. On the surface, they seem like reasonable tools, but in the absence of a rigorous, thorough supporting validity study, such a test may have little to no power to predict job performance.

The Impact of a Test with No Validity

For the sake of this hypothetical example, let’s assume the following:

1.      Unbeknownst to the hiring manager the selection test that he or she chooses to use has no or 0.0 correlation with job performance (In technical terms, a test with 0.0 correlation produces random results).

2.      100 job candidates are tested with this 0.0 correlation test

3.      The person doing the selection work understands the concept of selection ratios. He or she uses a 5:1 selection ratio to select the 20 candidates (20 percent) with the highest test scores.

 

What Usually Happens

 After the manager hires the top 20 percent, he or she assesses their actual job performance. He or she is surprised to discover that only 10 of these people (50 percent) are successful in learning and performing the job and the other ten people (50 percent) are not performing to the expected standard.

 

The Common Next Best Solution -- Become More Stringent

 The hiring manager decides that the problem can’t be the test because it came from a trusted source. Therefore, he or she decides that the course of action is to become more selective and use a higher selection ratio on the next batch of hires.

 

A decision is made to use a 10:1 selection ratio and only hire the top 10 percent or 10 applicants out of 100. (Recall that those 10 hires also will be the 10 who obtained the highest scores on the 0.0 validity selection test.)

 

Once again, the hiring manager assesses the actual job performance of the ten newest hires and is again surprised to discover that only five of the hires work out and five don’t.

Could a Test Deceive Us???

This cycle could continue until someone questions and evaluates the test.

 

Interestingly, many organizations with poor selection experience try to first look at more people with the same test. However, if the selection tools or what we refer to as the predictors they are using have low validity; Getting more selective or striving for a higher selection ratio does not help increase the “hit rate” or the number of hires that are good performers.

Assess & Understand Selection Tool Validity

 Many organizations are slow to recognize that they need to first find and use a valid selection tool or set of selection tools. They often become comfortable using convenient tools like one-on-one interviews or personality style assessments that are weak performance predictors.

Don’t Compromise Your Selection Efforts with Weak Predictors

 The key issue is the power of the predictor that is paired with the selection ratio. It doesn’t matter how many applicants you screen with a predictor or predictors of low validity. Organizations using predictors of low validity will be right only 50 percent of the time.

 

Selection Ratio Choices & Implications

The following is an illustration of how selection ratios work when paired with a valid predictor or predictors:

Selection Ratio Graphic

Hit Your Mark – A Selection Combination that Works

 You can only increase the hit rate by using a valid predictor, and you have to let it do its work by using a selection ratio greater than 1:1. (We coach our clients to strive for a 5:1 selection ratio. While a 5:1 ratio means there will be more work in terms of testing and interviewing, if a valid predictor is used, it provides insurance that you can reliably hire people who have the potential to learn and perform well on the job.)

 

 



Scott Valitchka has more than three decades of manufacturing-related organizational development, talent management, and process improvement experience with several consumer products companies.

Scott is a founding member and the Managing Partner of 15 Dots, LLC. Scott and the 15 Dots organization are committed to helping clients take the guesswork out of employee selection and consistently hire people with the ability to learn rapidly and perform well.

In his winter free time, Scott works as a professional ski instructor at Nordic Mountain, a Central Wisconsin ski area, and in the warmer months Scott enjoys teaching whitewater kayaking on Northern Wisconsin rivers.

Read more posts by Scott Valitchka

 Neenah, WI 54956
(804) 557-5033
Email 15dots