Offenders at the Heart of Evaluation

• Pages: 211-229
• DOI: https://doi.org/10.1108/S1474-7863(2012)0000013014
• Date: 21 May 2012
• Published date: 21 May 2012
• Author: Barry Kushner,Saville Kushner

OFFENDERS AT THE HEART

OF EVALUATION

Barry Kushner and Saville Kushner

ABSTRACT

This chapter looks at the role qualitative evaluation can play in the

external review of the Probation Service, the development of an

evaluation framework for ongoing assessment and how it can be used to

develop new elements of the service. How is this different from the use of

existing data by the service?

This takes us to the kind of information that is used by the probation

service, to make judgements about the effectiveness of its programmes and

the impact on offenders. The main contention is that this is in the main

quantitative data, and reports on levels of re-offending. The data is

standardised so that it can be used to make comparisons between different

types of sentence and criminal justice intervention.

Our contention is that this information is limited in what it says for two

main reasons. Firstly, quantitative data tends to report on impact, that is

whether an offender has committed another crime after engagement with

the probation service, or whether there are patterns of behaviour between

offences, individual circumstances and the likelihood or re-offending. In

short, the data is a snapshot of whether an offender has changed his or her

behaviour or not. However, this chapter will illustrate how quantitative

Perspectives on Evaluating Criminal Justice and Corrections

Advances in Program Evaluation, Volume 13, 211–229

Copyright r2012 by Emerald Group Publishing Limited

All rights of reproduction in any form reserved

ISSN: 1474-7863/doi:10.1108/S1474-7863(2012)0000013014

211

data misses an understanding of how behaviour changes and why

behaviour does not change. As a result, this leaves the service with a

limited understanding of how it is working.

Secondly, the chapter will argue that the existing probation framework

itself creates quite specific definitions of which data is relevant and which

data is not relevant. We will give examples of narratives that offenders

offer in group sessions that provide rich material about their lives,

pressures and offence. But evidence suggests that this information is not

used to inform programmes, re-assess and review an offender’s own

progression towards re-offending or a life without crime.

INTRODUCTION

This chapter looks at the role qualitative evaluation can play in the external

review of the Probation Service, the development of an evaluation

framework for ongoing assessment and how it can be used to develop new

practices in the sustainable reduction of offending. It is based on a

collaborative formative evaluation between the University of the West of

England and an English.

At present, the service relies substantially on quantitative – mostly

demographic – data for judgements about its effectiveness in reducing

offending. This data is standardised so that it can be used to make

comparisons between different types of sentence and criminal justice

intervention, as well as performance comparisons of different services and

regions. Evaluation is mostly confined to measures of service productivity.

However, this information is limited in what it says for two main reasons.

Firstly, the existing evaluative data focuses on impact, that is whether an

offender has committed another crime after engagement with the probation

service, or whether there are patterns of behaviour between offences,

individual circumstances and the likelihood of re-offending. The data is a

snapshot of whether an offender has changed his or her behaviour or not –

and, since it is standardised, it admits only of a limited range of possible

explanations and analyses of re-offending behaviours. This leaves the service

with a limited understanding of how it is working.

Secondly, the chapter will argue that the existing probation framework

itself creates specific definitions of which data is relevant and which data is

not relevant, that is pre-specifying and limiting the range of possible

explanations for recidivism or reduction in offending patterns. We will give

BARRY KUSHNER AND SAVILLE KUSHNER212