The Federal Uniform Guidelines on Employee Selection Procedures (1978)

• Author: Wayne F. Cascio,Herman Aguinis
• Published date: 01 September 2001
• Date: 01 September 2001
• DOI: http://doi.org/10.1177/0734371X0102100303
• Subject Matter: Articles

REVIEW OF PUBLIC PERSONNEL ADMINISTRATION / Fall 2001
Cascio, Aguinis / UPDATE ON SELECTED ISSUES
The Federal Uniform Guidelines on
Employee Selection Procedures (1978)
An Update on Selected Issues
WAYNE F. CASCIO
HERMAN AGUINIS
University of Colorado at Denver
The purpose of this article is to provide an update on a selected set of issues that
might be considered if and when the Uniform Guidelines on Employee Selec-
tion Procedures is revised. The following issues that have been subject to a con-
siderable number of research-based advances in the field of industrial and orga-
nizational psychology are discussed: (a) adverse impact, the four-fifths rule, and
statistical significance testing; (b) criterion measures; (c) cutoff scores; and
(d) differential prediction. In addition, implications for practice of research
findings in each of these areas are discussed.
SincethefederalUniformGuidelinesonEmployeeSelectionProcedureswere
issued in 1978, there have been a considerable number of research-based
advances in the field of industrial and organizational (I&O) psychology that
are relevant to various sections of the Guidelines. The purpose of this article is
not to provide an exhaustive discussion of all research that is remotely relevant
(e.g., cross-validation, validity generalization, alternative methods for esti-
mating reliability, the development of the O*Net). Rather, our goal is to dis-
cuss an admittedly selected set of issues that might be considered if and when
the Guidelines are revised. In particular, we will review research in four areas
that are particularly relevant to the Guidelines. After each section, we will
describe the practical implications of this body of research. The four areas that
we will discuss are the following (relevant sections of the Guidelines are
included in parentheses):
Authors’ Note: Correspondence regarding this article should be addressed to Wayne F. Cascio, Gradu-
ate School of Business Administration, University of Colorado at Denver, Campus Box 165, P.O. Box
173364, Denver, CO 80217-3364.
Review of Public Personnel Administration, Vol. 21, No. 3 Fall 2001 200-218
© 2001 Sage Publications
200



rop.sagepub.com
Downloaded from
at SAGE Publications on December 8, 2012

---

Cascio, Aguinis / UPDATE ON SELECTED ISSUES
201
1. Adverse impact, the four-fifths rule, and statistical significance testing. We will
discuss adverse impact and the four-fifths rule, including issues of statistical
power and statistical significance testing (relevant to § 1607.4, “Information
on Impact” and § 1607.14, “Technical Standards for Validity Studies”).
2. Criterion measures. We will discuss properties of criterion measures, including
dynamism of criteria, differences between typical and maximum perfor-
mance, and multidimensionality of criteria (relevant to § 1607.4, “Technical
Standards for Validity Studies”, Point B[3], “Criterion Measures”).
3. Cutoff scores. We will discuss professional issues, methods, and guidelines for
setting cutoff scores, including the latest legal pronouncements in this area
(relevant to § 1607.5, “General Standards for Validity Studies”, Point H, “Cut-
off Scores”).
4. Differential prediction. We will discuss recent findings in the area of differen-
tial prediction (i.e., fairness analysis) and factors affecting the accuracy of con-
clusions based on differential prediction analysis (relevant to § 1607.14,
“Technical Standards for Validity Studies”, Point B[8], “Fairness”).
Another way to view the above outline is in terms of practical questions
related to the implementation of the Guidelines (1978). That is, one begins
by asking, Is there a basis for enforcement of the Guidelines (our Section 1)?
If yes, is there overall evidence of validity? Specifically, if an empirical
research design is used, is there justification for the measures of perfor-
mance, that is, criterion measures, used (our Section 2)? Is there justifica-
tion for any cutoff scores used (our Section 3)? and If there is overall evi-
dence of validity, is there specific evidence of unfairness for subgroups (our
Section 4)?
ADVERSE IMPACT, THE FOUR-FIFTHS RULE,
AND STATISTICAL SIGNIFICANCE TESTING
Section 1607.4, “Information on Impact”, does not include any discus-
sion of statistical power or statistical significance testing in evaluating dif-
ferences in pass rates or selection rates from two or more subgroups of indi-
viduals. On this issue, the Guidelines (1978) recommends an arbitrary rule
of thumb known as the four-fifths rule. This rule states that a difference in
pass rates between two subgroups is not generally considered substantial if
the pass rate for one subgroup is at least four-fifths (80%) of the pass rate for
the higher subgroup. As Shoben (1978) noted,
The four-fifths rule is an ill-conceived resolution of the problem of assessing
the substantiality of pass or acceptance rate differences. It will produce
anomalous results in certain cases because it fails to take account of differ-



rop.sagepub.com
Downloaded from
at SAGE Publications on December 8, 2012

---

202
REVIEW OF PUBLIC PERSONNEL ADMINISTRATION / Fall 2001
ences in sampling size. It also neglects the magnitude of differences in pass
rates by considering only the ratio of the two rates. (pp. 805-806)
Shoben (1978) argued that the flaws in the four-fifths rule can be elimi-
nated by replacing it with a test of the statistical significance of differences
in pass rate proportions. Such a test takes into consideration the size of the
sample and the magnitude of the differences in pass rates.
In fact, a disparity in pass or acceptance rates has one of three possible
causes, as the United States Court of Appeals (D.C. Circuit) in Palmer v.
Shultz (1987) pointed out. One, the disparity may be a product of unlawful
discrimination. Two, the disparity may have a legitimate and nondiscrimina-
tory cause. Three, the disparity may simply be a product of chance.
A statistical analysis of a disparity in selection rates can reveal the probability
that the disparity is merely a random deviation from perfectly equal selec-
tion rates. Statistics, however, can not entirely rule out the possibility that
chance caused the disparity. Nor can statistics determine, if chance is an
unlikely explanation, whether the more probable cause was intentional dis-
crimination or a legitimate non-discriminatory factor in the selection pro-
cess (p. 11, italics in original).
Title VII nevertheless provides that if the disparity between selection
rates . . . is sufficiently large so that the probability that the disparities
resulted from chance is sufficiently small, then a court will infer from the
numbers alone that, more likely than not, the disparity was a product of
unlawful discrimination—unless the defendant can introduce evidence of a
nondiscriminatory explanation for the disparity or can rebut the inference
of discrimination in some other way. (pp. 11-12)
The court (Palmer, 1987) recommended the use of a .05 level of statisti-
cal significance, and two-tailed tests, in which “a statistically significant
deviation in either direction from an equality in selection rates would con-
stitute a prima facie case of unlawful discrimination” (p. 21).
In spite of the court’s endorsement, null hypothesis significance testing
(e.g., a test of a null hypothesis of equality of selection rates across sub-
groups) has been, and still is, a topic of heated debate in the scientific com-
munity (e.g., Aguinis, 1995; Chow, 1988, 1996; Cohen, 1994; Cortina &
Folger, 1998; Murphy, in press; Murphy & Myors, 1998). Researchers have
written extensively on the purpose, meaning, and use of significance test-
ing. Some argue that significance testing is useful (e.g., Wainer, 1999),
whereas others believe that it is misleading and should be discontinued
(e.g., Schmidt, 1996). Next, we frame the issues of meaning, purpose, and
use of significance testing within the context of adverse impact analysis (i.e.,



rop.sagepub.com
Downloaded from
at SAGE Publications on December 8, 2012

---

Cascio, Aguinis / UPDATE ON SELECTED ISSUES
203
a significance test of the null hypothesis that the selection rates are equal
across subgroups).
Purpose of Significance Testing
The purpose of significance testing is to determine whether a finding of
inequality of selection rates in a sample of applicants can be explained by
chance alone (i.e., sample fluctuations). Significance testing is used only
when we use samples of applicants to make inferences regarding populations
of applicants. Significance testing is not needed when we do not wish to make
inferences from samples to populations. For instance, assume the admittedly
unrealistic situation in which we use a selection instrument one time only
with one sample of applicants only. Assume that Subgroup 1 (e.g., men) has a
selection rate of .50 (i.e., 50% of men are given a job offer), and Subgroup 2
(e.g., women) has a selection rate of .60 (i.e., 40% of women are given a job
offer ). The conclusion is that these rates are different. In other words, men
are selected at a greater rate as compared to women (i.e., 50% > 40%). Now
assume the more typical situation in which we have a sample of applicants,
but we are planning on using the selection procedure in the future with other
applicants. In this case, we need to infer whether the .10 difference in the
selection rates in our sample can be explained by chance alone (i.e., sample
fluctuations) or by a robust finding. In this situation, in which we make infer-
ences from samples...