Connecting the Dots Blog

A Puzzling Scenario - Part I

A Hypothetical Situation to Illustrate Selection Principles

 This is the first in a three-part blog series.  I will be sharing the relationship that exists between validity and selection ratios. This is important because an understanding of these principles ensures the operation of sound selection systems.  In this first case, I want to focus on how the use of a perfect selection test can impact selection outcomes.

 

Assume that the applicants for a job are normally distributed (bell-shaped curve)  around ultimate job performance. (See figure 1.)

 

Some applicants will be above average performers, some will be below average performers, and most will fall into the average range. That assumption is reasonable.

 

Many organizations believe that they have formed robust applicant pools. However, I have often found that those pools are normally distributed meaning that a few candidates in the pool are slightly better than average (at best), and that a lot are below average. 

Job Applicant Normal Distribution Figure 1

 

Here is an example of what I mean:

If an applicant pool then has 100 people in it, and they are normally distributed around ultimate job performance, then 50% will be above average performers. The question is: which 50%? The identification process is the object of selection systems. There are several tools to help measure a person’s capacity to succeed in a company. Let’s consider testing.

 

Tests and a Good Performance Evaluation System

Let’s first assume that you have a normally distributed applicant pool, and assume that you have a test that has perfect validity of 1.0.

 

Further assume that your performance evaluation system is a 9-point system using 9 for the top performer, 5 for an average performer, and 1 for the lowest performer. Another assumption being that scores on the test range from 1 to 9 with 9 being the highest, 5 being average, and 1 being the lowest. (See figure 2.)

 

Selection Rating Scale

Figure 2

 Choose Your Selection Ratio

​You test the entire group of 100 people with the perfectly valid test, still not knowing who the best will be, but assuming (correctly) that the highest scorers on the test will be ultimately be the best performers. You select all the 9s first. You then may select 8s knowing that they will still be good performers, just not quite as good as the 9s, then you choose 7s , etc. You would eventually have to stop somewhere, but the question is where? A 9 is in the top 2% of a normal distribution. All 8s and 9s are in the top 10% and all 7s, 8s, and 9s are in the top 20%. A 6 is in the top 1/3rd, a 5 is the top half, and so on.  You may decide to stop at the 7 which belongs in the top 20%.

 

Selection Ratio Defined

If you follow the above scenario as a standard practice and only hire those who scored a 7, 8, or 9, you are establishing what is referred to as a selection ratio. A selection ratio is the number of people tested: relative to the number of people selected. Assuming that position, taking the top 20% (the 7s, 8s, and 9s) is a 5:1 selection ratio, and taking the top 1/3rd is a 3:1 ratio and so on.

 

Selection Ratio Outcomes

If you take the top 20% or 20 candidates from the 100, you will have all above average performers spread throughout the group. If you take the top 33, you’ll still have all above average performers, but the group’s overall performance will be lower. It’s obvious that if you take the top two-thirds or 67 people, you’ll systematically select a number of below average performers and the group’s overall performance drops further.

 

Taking it to the Limit -- What’s the Answer?

If you select all 100, you’d get 50 above average performers (some far above average) and 50 below average performers (some far below average). Think about this for a minute. You have a perfect selection system, but you didn’t let it do its work! You selected everyone you tested. A major question always persists being, “What’s your hit rate when you have a perfect system and you select everyone?” The answer being, “Your hit rate is 50%.” This answer is baffling to most people until they understand the relationship between validity and selection ratio.

 

Even with a perfect system, your hit rate goes down as your selection ratio goes down, due to the fact that applicants are normally distributed and they “present” themselves for selection in that distribution.

 

The Bottom line

In order to accomplish your selection objective of having applicants who are likely to be good performers, you have to have a selection ratio that’s greater than 1:1. The greater this is, the better.

 

Stay tuned for part II in this three-part blog series. I will share an even more puzzling outcome when you use a test that has no validity.



Joe Nowlin earned his doctorate from the Indiana University Kelley School of Business.

During the past 40 years, he has designed, tested, and installed proprietary and unmatched selection procedures for manufacturing organizations, transforming and revolutionizing the way they do business, the level of success in hiring personnel and the methods that companies can use to replicate his results.

He has now adapted these proprietary systems for use in medium-sized and smaller organizations.

 

Read more posts by Joe Nowlin

 Neenah, WI 54956
(804) 557-5033
Email 15dots