Risk-Appraisal Versus Self-Report in the Prediction of Criminal Justice Outcomes
| Published date | 01 June 2006 |
| DOI | 10.1177/0093854805284409 |
| Date | 01 June 2006 |
| Author | Glenn D. Walters |
| Subject Matter | Articles |
CRIMINAL JUSTICE AND BEHAVIOR
Walters / RISK-APPRAISAL VERSUS SELF-REPORT
RISK-APPRAISAL VERSUS
SELF-REPORT IN THE PREDICTION
OF CRIMINAL JUSTICE OUTCOMES
A Meta-Analysis
GLENN D. WALTERS
Federal Correctional Institution
Twenty-seven individual pairs of effect sizes from 22 prospective studies employing one or more
of the following five risk-appraisal procedures: Historical-Clinical-Risk Scales (HCR–20), Life-
style Criminality Screening Form (LCSF), Level of Service-Inventory (LSI), Psychopathy
Checklist (PCL), Violence Risk Appraisal Guide (VRAG), and one or more self-report measures
were subjected to meta-analysis. Although risk-appraisal procedures displayed an advantage
over self-report measures in recidivism prediction, the two methods produced comparable
results when the meta-analysis was restricted to investigations using content-relevant self-report
predictors. Incremental validity analysis of 72 risk-appraisal/self-report contrasts revealed that
both sets of measures accounted for criminal justice outcomes beyond the variance attributable to
the alternate method.
Keywords: risk assessment; self-report measures; meta-analysis; prediction; Psychopathy
Checklist
Skepticism abounds over the prospect of incorporating offender
self-report into the criminal justice decision-making appara-
tus. Such cynicism is understandable given the fact that some offend-
ers lie and scheme in an attempt to avoid responsibility for their crimi-
nal actions. Risk-appraisal procedures like the Level of Service
Inventory—Revised (LSI–R; Andrews & Bonta, 1995) and Psychop-
athy Checklist—Revised (PCL–R; Hare, 1991) have replaced self-
CRIMINAL JUSTICE AND BEHAVIOR, Vol. 33 No. 3, June 2006 279-304
DOI: 10.1177/0093854805284409
© 2006 American Association for Correctional and Forensic Psychology
279
280
CRIMINAL JUSTICE AND BEHAVIOR
report inventories as the method of choice for clinicians on the
strength of the belief that risk-appraisal measures are less susceptible
to subterfuge. It should therefore come as no surprise that research
interest in risk appraisal greatly exceeds research interest in offender
self-report. Readers may recall that three special issues on risk assess-
ment have been published in Criminal Justice and Behavior in recent
years (Heilbrun, 2002; Rice & Harris, 2001; Simourd, 2002); how-
ever, only a handful of studies on offender self-report worked their
way into this journal during the same time period. Most criminal jus-
tice journals, in fact, devote substantially more time and space to risk-
appraisal procedures than they do to offender self-report, a finding
that is readily discernable from a cursory review of major criminal jus-
tice periodicals. The question posed by this article is whether proce-
dures fashioned specifically for risk appraisal perform at a level com-
mensurate with their overrepresentation in the literature vis-à-vis
procedures that rely exclusively on offender self-report.1
Edens, Hart, Johnson, Johnson, and Olver (2000) have explored the
reasons behind correctional and forensic researchers’ apparent lack of
interest in assessment based on self-report. First, there is the problem
of response distortion for the purpose of gaining privileges or avoid-
ing consequences. There will always be offenders who feign psycho-
logical problems in hopes of manipulating a transfer to a medical
facility or who assume a courteous facade in an effort to deceive the
parole board into granting them an early release. Just the same, several
of the self-report measures included in this meta-analysis come
equipped with validity or response style scales. Validity scales are
found on the Minnesota Multiphasic Personality Inventory (MMPI;
Hathaway & McKinley, 1967), Personality Assessment Inventory
(PAI; Morey, 1991), and Psychological Inventory of Criminal Think-
AUTHOR’S NOTE: The author would like to thank James Bonta, John F. Edens,
Paul Gendreau, Claire Goggin, Grant T. Harris, Kim Harrison, Daryl Kroner, Alex-
ander Loucks, William T. Palmer, Richard Rogers, and Jennifer L. Skeem for their
assistance in identifying studies and furnishing data/correlations unavailable from
published sources. The assertions and opinions contained herein are the private views
of the author and should not be construed as official or as reflecting the views of the
Federal Bureau of Prisons or the United States Department of Justice. Address all
correspondence to Glenn D. Walters, Psychology Services, FCI-Schuylkill, P.O. Box
700, Minersville, PA 17954-0700; e-mail: gwalters@bop.gov.
Walters / RISK-APPRAISAL VERSUS SELF-REPORT 281
ing Styles (PICTS; Walters, 1995). Besides detecting simulated test
results (Morey, 1991; Walters, 2001), validity scales and indices are
sensitive to response exaggeration secondary to emotional distress or
situational demand (Walters, White, & Greene, 1988). Self-report
measures can be structured to identify and compensate for respondent
duplicity, but vulnerability to deceit is not the only facet of offender
self-report that concerns researchers.
Self-report surveys have also been reproached for weak content
validity (Edens et al., 2000). Whereas such reservations are fitting
when applied to empirically derived measures like the MMPI, they are
less relevant to more theoretically informed instruments like the PAI,
PICTS, and Criminal Sentiments Scale (CSS; Andrews & Bonta,
1995). Morey (1991) went to great lengths to model the Antisocial
Features (ANT) scale of the PAI after contemporary research and the-
ory on antisocial personality and psychopathy. The PICTS, on the
other hand, derives from a theory of criminal cognition organized
around eight thinking styles (mollification, cutoff, entitlement, power
orientation, sentimentality, superoptimism, cognitive indolence, dis-
continuity). Each thinking style is represented by a separate PICTS
scale with items designed to sample major aspects of the purported
thinking style (Walters, 1995). Attempts were also made to keep the
CSS as theoretically pure as possible by limiting scale content to items
that described antisocial attitudes and beliefs construed as instrumen-
tal in supporting and facilitating criminal activity (Simourd & Olver,
2002). Another test of criminal attitudes and beliefs, the Self-
Appraisal Questionnaire (SAQ; Loza, Dhaliwal, Kroner, & Loza-
Fanous, 2000), was devised as a means of appraising six areas
identified by research as highly prognostic of recidivism.
Self-report instruments have also been faulted for limited applica-
bility in the sense that respondents must meet minimum reading
requirements to complete a self-report survey. There is no getting
around the fact that literacy is required to competently answer the
questions on a self-report inventory. Nonetheless, most offenders are
capable of reading at or above an upper-primary-school level, and
between 84% and 89% of state, federal, and jail inmates possess at
least an eighth-grade education (Harlow, 2003). The majority of self-
report scales included in this meta-analysis demand a sixth-grade
reading level, although a few, most notably the PAI, can be completed
282
CRIMINAL JUSTICE AND BEHAVIOR
by individuals functioning at a fourth-grade reading level (Morey,
1991), and recent advances in the use of nonverbal response formats
(Paunonen, Jackson, Trzebinski, & Forsterling, 1992) have made self-
report measures more accessible to a larger audience of respondents.
Hence, although reading ability may limit the applicability of self-
report measures relative to risk-appraisal strategies, most offenders
can read and comprehend well enough to answer the majority of ques-
tions found on a self-report inventory. Whether these individuals pos-
sess sufficient insight to effectively respond to the questions is another
criticism that has been leveled against the widespread use of self-
report surveys to answer correctional and forensic questions (Edens
et al., 2000). In any event, if self-report measures achieve results anal-
ogous to those attained with a risk-appraisal procedure, then the
question of insight or psychological mindedness becomes moot.
The principal purpose of this meta-analysis was to ascertain whether
traditional assumptions about the superiority of risk-appraisal proce-
dures relative to self-report measures in the prediction of criminal jus-
tice outcome hold up under empirical scrutiny; criminal justice out-
come is defined as disciplinary adjustment in prison, recidivism on
release, and violence both in prison and in the community. A meta-
analysis was performed on all prospective institutional adjustment
and recidivism studies, rendering a direct comparison between one or
more of the following five risk-appraisal procedures: the HCR–20
(Webster, Eaves, Douglas, & Wintrup, 1995), LCSF (Walters, White,
& Denney, 1991), LSI, PCL,2 VRAG (Harris, Rice, & Quinsey, 1993),
and one or more self-report scales. First, similar levels of predictive
validity were anticipated such that the 95% confidence intervals for
the risk-appraisal and self-report measures would overlap and the
combined Z scores converge, thereby signaling a lack of meaningful
differentiation between the two methods. Second, studies comparing
risk-appraisal procedures and content-relevant self-report surveys
were examined in an effort to determine whether restricting the self-
report method to content-relevant procedures had any effect on the
relative standing of risk appraisal and self-report in the prognostica-
tion of criminal justice outcome. Third, incremental validity analyses
were...
To continue reading
Request your trial