The authors examined the thinking processes in the use of a decision support system (DSS) by social workers in a human services agency to determine whether they used the system to improve their case reasoning. Information was obtained from in-depth interviews with eight social workers who used a DSS in their work and from content analysis of "narrative justifications" appended to 1,074 decisions that differed from those recommended by the DSS. Findings show that the social workers used the DSS in a perfunctory manner in typical cases, but as an aid to thinking and reflection in atypical cases, in which they were uncertain of how to decide. On the whole, the social workers showed little interest in the DSS's recommendations, but reported that the process of entering data and answering the model's questions about the case were useful when the case was atypical. The authors suggest that computer support systems should be designed to help workers think through atypical cases.
KEY WORDS: decision support systems; evaluation; probation services; professional judgment; professional reasoning
The use of computerized decision support systems (DSSs) has been advocated to increase equity and reduce error variance in human services decisions (Proctor, 2002; Schoech, 1999). A widely used type of DSS works by matching characteristics of current cases to decision criteria established through statistical analysis of a large number of previous cases and the decisions made in them. The DSS is programmed with questions to elicit the characterstics of the current case from the workers and, after processing the entries, provides a normative decision based on the decisions made by others under the same conditions. The premise is that the normative decision is more equitable and less error prone than human judgment (Benbenishty, 1992; Quartararo, Glasziou, & Kerr, 1995; Rosen, Proctor, Morrow-Howell, & Staudt, 1995).
Along with equity and error reduction, writers also have touted the DSS as an aide to improving professional reasoning (for example, Benbenishty, 1992; Carlson, 1993; Linn, Schneider, McCreery, & Kasab, 1998; Schuerman, 1987). The assumption, largely unstated, is that human judgment must be included in the decision-making process. Thus, virtually all DSSs allow workers to diverge from the computer recommendations. Although writers do not expand on how the DSS is supposed to improve reasoning, two ways may be suggested. One is that the questions posed by the computer may lead workers to consider in a structured manner all the relevant aspects of their cases and the available interventions and outcomes. The other is that the dissonance created by non-normative decisions may lead workers to rethink their decisions, especially in cases where the system requires them to justify divergent decisions.
Most studies conducted on DSSs have dealt with the first aim, focusing on the design and efficacy of the models. These studies point to the contribution of DSSs to the attainment of equity in a variety of tasks with a range of populations: matching children with alternative care facilities (Schwab, Bruce, & McRoy, 1986), assessing risk of psychiatric rehospitalization (Patterson & Cloud, 1999), selecting treatment for substance abuse (Bencel et al., 1998), discharging soldiers from military service (Benbenishty & Treistman, 1998), making court recommendations on juvenile offenders (Savaya, Monnickendam, & Waysman, 2000), and (under such rubrics as "guideline-based decision support") assisting doctors with medical decisions (Goldstein et al., 2000; Kohn, Corrigan, & Donaldson, 2002).
The present study focuses on the second aim--the improvement of professional reasoning.We examined the thinking processes that workers brought to bear on their use of the DSS. We first examined workers' thinking when their decisions were different from DSS-recommended decision to determine whether and how discrepant recommendations stimulate thought and reflection. Such understanding is important in view of the increasing recognition of the need for practice evaluation in social work (Rosen, 1992) and the potential of DSSs to facilitate such evaluation.
Most studies of DSSs have been conducted on prototypes or experimental applications, largely because of the rarity of DSSs in routine use. In our study, the workers were in a setting and situation in which the DSS was an integral part of the social workers' concept of treatment. The DSS had been operative in the human services agency for almost 10 years, and we examined workers who were known to use the system, at least nominally, as part of their regular work routine.
DSS FOR THE YOUTH PROBATION SERVICES IN ISRAEL: DESIGN AND PROCESSING
The article reports on the second part of an evaluation of a DSS used in the Youth Probation Services in Israel, a nationwide government human services agency. The first part of the evaluation examined the extent to which the workers used the system, the obstacles to routine use, and the success of the model (Savaya et al., 2000).
Youth probation officers in the Juvenile Probation Services in Israel are responsible for making placement and service recommendations to the courts with regard to the treatment of juvenile offenders. In accordance with government regulations, these officers are licensed social workers,with at least a BSW. Following British practice (Nellis, 1999), they are part of the social welfare system--not officers of the court, as in the United States--and their task is to treat rather than supervise the offenders.
To design the system, data from more than 1,000 cases previously handled by the Youth Probation Service were analyzed through a policy-capturing process (Larcker & Lessing, 1983); a multiple linear regression identified eight key variables that predicted the social workers' recommendations. They formed the basis of the model that was incorporated in the computer system to provide the workers with recommendations reflecting normative decisions made under the same conditions.
Cases are processed in three stages. In the first stage, the DSS presents the social workers with 80 fixed-choice questions on the case at hand. Some of the questions are objective (for example, juvenile's age, gender, offense, family composition, previous offenses), others are evaluative (for example, assessments of the seriousness of the offense, the likelihood that the offense will be repeated, the juvenile's susceptibility to negative influences, and so forth). After entering all the answers, the social worker selects one of five standard recommendations that regulations permit them to make to the courts. In the second stage, the program runs the data through the statistical model and generates a recommendation. If the two recommendations are identical, the process is concluded. If they differ, the social worker must decide whether to accept or reject the DSS's suggestion and enter the recommendation that he or she ultimately chooses. Finally, the system provides two blank lines in which the social worker must explain, in free narrative, why he or she rejected the DSS recommendation or changed their mind and accepted it.
Goals of the DSS and of this Study
The DSS had two main goals. The first was to make the social workers' recommendations more equitable (Stagner & Johnson, 1994). Examination showed that 71% of the DSS recommendations were equitable (that is, normative), but only 61% of the social workers' recommendations were equitable (Shapira, 1990).
The second goal was to improve the social workers' decision reasoning by inducing methodical consideration of all the relevant data and by encouraging critical review and self-examination (Shapira, 1990). We examined the thinking that social workers brought to bear on their use of the DSS: Was their thinking affected by their answering the questions and entering the data? Did they review and rethink recommendations that differed from those of the DSS, or did they use the system perfunctorily? Under which circumstances did they review and rethink, and under which circumstances did they not?
We used a multifaceted qualitative research strategy with information obtained from content analysis of the social workers' narrative justifications and in-depth interviews.
During the period of the study (1994-1996), 2,757 cases were entered into the DSS. In 1,088 cases (39%), the social workers' decisions differed from the DSS recommendation. For the content analysis, the 1,088 narrative justifications, which were originally rendered in free text, were transferred to EXCEL and then to SPSS; each word was assigned to a separate cell for counting and classification. We then cleaned the statements; 14 were dumped as incomprehensible (Jones, Miles, & Read, 1996), leaving 1,074 narrative justifications.
To classify the statements into analytic categories, we adopted an...