Are Protective Factors Valid Constructs? Interrater Reliability And Construct Validity Of Proposed Protective Factors Against Sexual Reoffending

• Author: David Thornton,Sharon M. Kelley,Gwenda M. Willis
• Published date: 01 November 2020
• DOI: 10.1177/0093854820941039
• Date: 01 November 2020
• Subject Matter: Articles

CRIMINAL JUSTICE AND BEHAVIOR, 2020, Vol. 47, No. 11, November 2020, 1448 –1467.

DOI: https://doi.org/10.1177/0093854820941039

Article reuse guidelines: sagepub.com/journals-permissions

© 2020 International Association for Correctional and Forensic Psychology

1448

ARE PROTECTIVE FACTORS VALID

CONSTRUCTS?

Interrater Reliability And Construct Validity Of

Proposed Protective Factors Against Sexual

Reoffending

GWENDA M. WILLIS

The University of Auckland

SHARON M. KELLEY

Sand Ridge Secure Treatment Center

DAVID THORNTON

Forensic Assessment, Training, & Research, LLC

Most sexual recidivism risk assessment tools focus primarily on risk factors and deficits without consideration for strengths

or protective factors which might mitigate reoffense risk. The current study is the first in a research program designed to

develop and validate the Structured Assessment of PROtective Factors for violence risk—Sexual Offence version

(SAPROF-SO), a measure of protective factors against sexual reoffending. The study aimed to test interrater reliability and

construct validity of the SAPROF-SO with a high-risk (n = 40) and routine (n = 40) sample. Interrater reliability between

three independent raters was generally good to excellent for the SAPROF-SO domain and Total scores across both samples

and compared favorably with validated measures of dynamic risk. Moreover, the SAPROF-SO demonstrated construct valid-

ity and was moderately independent of existing measures of risk. Findings open the door for a more balanced, strengths-

based, and accurate approach to recidivism risk assessment.

Keywords: protective factors; sexual offending; SAPROF; risk assessment; desistance

Assessment technology available to professionals working with men known to have

committed sex offenses has yet to catch up with strengths-based approaches to treat-

ment and desistance research (Laws & Ward, 2011; Marshall et al., 2017). Commonly used

instruments including the Static-99R (Helmus, Thornton, et al., 2012) and Stable-2007

AUTHORS’ NOTE: The authors acknowledge Melissa Adam and Shane Brown who coded the routine sample

with the first author for interrater reliability as part of their doctoral and master’s theses, respectively. The

opinions are those of the authors and not necessarily those of Sand Ridge Secure Treatment Center or the New

Zealand Department of Corrections. This research was supported by a Rutherford Discovery Fellowship

awarded to Gwenda M. Willis. Correspondence concerning this article should be addressed to Gwenda M.

Willis, School of Psychology, The University of Auckland, Private Bag 92019, Auckland 1142, New Zealand;

e-mail: g.willis@auckland.ac.nz.

941039CJBXXX10.1177/0093854820941039Criminal Justice and BehaviorWillis et al. / Reliability and Validity of Protective Factors

research-article2020

Willis et al. / RELIABILITY AND VALIDITY OF PROTECTIVE FACTORS 1449

(Fernandez et al., 2014) are composed primarily of risk factors and understood as measures

of risk. Their focus on risk factors and deficits creates a tension between the developing

ethos of treatment and the practice of assessment, making it harder for one to inform the

other. The development of measures of protective factors represents a promising way for-

ward. The current article describes the development and initial research with a new instru-

ment, the Structured Assessment of PROtective Factors for violence risk—Sexual Offence

version (SAPROF-SO), designed to support the systematic and structured assessment of

protective factors relevant to sexual recidivism risk.

For our purposes, and in line with other researchers’ definitions (e.g., de Vogel et al., 2012),

protective factors can be defined as those factors that are theoretically or empirically associ-

ated with reduced rates of sexual or violent recidivism in individuals with at least one appre-

hension for a sexual offense as an adult. They must signal the presence of a strength, not

merely the absence of a risk factor or deficit (de Vries Robbé, Mann et al., 2015). Some pro-

tective factors may reflect the opposing, positive pole of a risk factor, and both the positive

and negative can coexist (e.g., prosocial and antisocial peers, prosocial and deviant sexual

interests; de Vries Robbé, Mann et al., 2015). Other proposed protective factors reflect vari-

ables independent of known risk factors (e.g., medication, life goals; de Vogel et al., 2012).

Notwithstanding whether protective factors represent anything new, as highlighted by Helmus

(2018), “. . .the possibility remains that they are measuring existing factors better” (p. 5). In

this initial article, we describe the SAPROF-SO development process and report initial

research findings seeking to answer the question, are protective factors, as measured by the

SAPROF-SO, valid constructs? We have deliberately set a low bar for an affirmative answer

to this question. Specifically, we address, (a) whether the SAPROF-SO assesses protective

factors in a reliable way, (b) whether the SAPROF-SO’s protective factors show convergent

and divergent validity, and (c) whether the SAPROF-SO is measuring something beyond

established risk assessment instruments. If the SAPROF-SO fails to produce reliable and

independent variance, it could be argued that it lacks novelty, even though its positive reframe

of variance captured by existing instruments might still be clinically useful. Questions of pre-

dictive utility will be investigated in future studies and are not the focus of the present study.

RELEVANCE TO RISK ASSESSMENT

Recidivism risk assessments inform a variety of important decisions in forensic and cor-

rectional settings including the form and intensity of treatment and supervision, as well as

judicial decisions such as readiness for discharge or parole. Accordingly, improving the pre-

dictive accuracy of risk assessment practices represents an ongoing research priority. Although

contemporary tools perform better than their predecessors, they are not able to identify groups

with truly high sexual recidivism rates. For example, Fernandez et al. (2014) reported that

after 5 years, the majority of individuals assessed in the high or very high priority categories

for sexual reoffending based on combined Static-99R/Stable-2007 scores were not charged or

reconvicted for a new sexual offense. Using the Violence Risk Scale—Sexual Offense version

(VRS-SO), Olver et al. (2007) found that after 10 years, 70% of men assessed in the highest

risk category reoffended with a sexual offense, and in an independent validation study of the

VRS-SO, Beggs and Grace (2010) found that 56.2% sexually reoffended after 12 years. The

presence of protective factors might reasonably differentiate individuals assessed with a

greater than average risk to reoffend who do not reoffend, compared with those who do.