Evaluating Training for New Government Officials: A Case Study Using the Success Case Method
| Date | 01 December 2017 |
| Author | Jaeeun Lee,Wooseok Kim,Dongwon Jeon,Chan Lee |
| DOI | 10.1177/0091026017730382 |
| Published date | 01 December 2017 |
| Subject Matter | Articles |
research-article2017
Article
Public Personnel Management
2017, Vol. 46(4) 419 –444
Evaluating Training for New
© The Author(s) 2017
Reprints and permissions:
Government Officials: A Case sagepub.com/journalsPermissions.nav
https://doi.org/10.1177/0091026017730382
DOI: 10.1177/0091026017730382
journals.sagepub.com/home/ppm
Study Using the Success Case
Method
Chan Lee1, Dongwon Jeon2, Wooseok Kim2,
and Jaeeun Lee3
Abstract
This study evaluates the training program for newly appointed government officials
in Korea and presents the effectiveness of the success case method (SCM) technique
for such an evaluation. The training program chosen as the case study is administered
by the training institute for public officials of Korea. The SCM technique was used to
evaluate this program and to draw implications that could be applied to the design
of future training programs. A survey was conducted targeting 321 trainees who
completed the program for newly appointed government officials, and 12 trainees
were selected who showed a high level of training transfer measures for the
qualitative examination. Through the analysis of semistructured interviews, the study
deduced the factors that influenced effective training transfer. The results illustrate
that the factors that lead to successful application of the training are 15 success
factors categorized in the trainee’s individual characteristics, the training design, and
the trainee’s work environment. This study provides implications that can be useful
in the workplace, including the application of the SCM in evaluating training programs
in the public sector, and the factors that should be considered for effective training
transfer among newly appointed public officials.
Keywords
success case method, training transfer, training evaluation, public sector training, case
study
1Seoul National University, Korea
2Dohow Consulting, Seoul, Korea
3Konkuk University, Chungju-si, Korea
Corresponding Author:
Jaeeun Lee, Konkuk University, 268, Chungwon-daero, Chungju-si, Chungcheongbuk-do, 27478, KOREA
Email: violet80119@gmail.com
420
Public Personnel Management 46(4)
Introduction
A large number of organizations have recognized that developing employee talent is
the only way to maintain competitiveness, especially in environments of rapid and
dramatic change, and are thereby expanding investment in human resource develop-
ment (C. Lee, Lee, Lee, & Park, 2014). This emphasis on training does not exclude the
public sector. As the public sector workplace continues to confront a range of chal-
lenges, organizations must commit to the strategic development of employees’ knowl-
edge, skills, and abilities (Pynes, 2013). The discussion on the need for training has a
long history. About 40 years ago, Mikesell, Wilson, and Lawther (1975) noted that
training could improve many government activities and achieve better job perfor-
mance in a reasonably short period of time.
One of the important issues in training and development is to determine the effects
of training intervention. This means, it is critical to identify the effectiveness of the
training program as it provides the key information for decision makers around further
training investment (van Buuren & Edelenbos, 2013). Many scholars have discussed
the training effect and how to evaluate it. For example, Kirkpatrick’s (1976) four-level
evaluation has been used extensively in many training areas and several studies on
public sector training evaluations utilize it as well (e.g., Bjornberg, DellCioppia, &
Tanzer, 2002; Getha-Taylor, Fowles, Silvia, & Meritt, 2015; Olivero, Bane, &
Kopelman, 1997).
One of the most important concepts in evaluating the effectiveness of a training
program is training transfer, which is also a major concern in the Kirkpatrick model’s
behavior level. Generally, training transfer is defined as the application of knowledge,
skills, and attitudes learned from training on the job and subsequent maintenance of
them over a certain period of time (Baldwin & Ford, 1988; Cheng & Ho, 2001).
Successful training transfer has been recognized as the means to reduce the gap
between employee training and actual performance (C. Lee et al., 2014). In a training
program for public officials, the evaluation goal for training transfer is the same—to
confirm the degree to which trainees apply effectively to their work after obtaining the
knowledge and skills in the training program—and is considered of great importance,
along with the need for a methodical approach and systematic analysis.
In this context, traditional training evaluation models, such as Kirkpatrick’s (1976),
have been criticized because they only focus on the performance of the “training pro-
gram” and not on various environmental factors impacting performance. Bates (2004)
pointed out that the Kirkpatrick model excluded and simplified the various environ-
mental factors surrounding the training program, resulting in popularity; however, it
failed to systematically explore the various factors that affect the effectiveness of a
training program. To address a general frustration with such traditional evaluation mod-
els, the success case method (SCM) was developed. SCM is a process for evaluating the
business effect of training that is aligned with and addresses the organization’s strategy
(Brinkerhoff, 2003). The SCM identifies the critical success and failure factors of train-
ing transfer for the overall analysis, and has been useful in qualitatively estimating and
validating the effectiveness of many training programs (Choi & Lee, 2011).
Lee et al.
421
Although the SCM technique has been applied to a wide range of areas, there are
scant cases where it is used to evaluate training programs in public sector. One reason
for the difficulty in applying SCM to public sector training programs was that SCM
focuses on specific business outputs. However, recently, research has been conducted
on educational officers applying the SCM process in the public sector in Korea (Lee,
Choi, & Park, 2013). Research has also been conducted on U.S. health care officials
(Medina et al., 2015) and on nonprofit social service organizations (Coryn, Schröter,
& Hanssen, 2009).
This study utilizes the SCM technique to examine a case study of a specific training
program for public servants. The study aims to evaluate various aspects of this training
program and explore the factors that influence the effectiveness of it for newly
appointed government officials in Korea. Through this research, the study attempts to
increase the applicability of SCM to the evaluation of training programs for public
servants and to suggest various ways that can improve training transfer.
Literature Review
Training Program Evaluation in the Public Sector
A training program evaluation is the process of assessing the value of the program, and
this can be done by evaluating the program’s results, namely, the program’s perfor-
mance, and by evaluating the program’s whole process that involves the program’s
intention, content, and implementation, or by using both methods simultaneously. In
the existing literature, the specific definition of “training program evaluation” may
vary among researchers, but there is a shared concept of the training program evalua-
tion as the process of identifying the program’s value, achievement of objective, and
effectiveness.
The evaluation includes collecting information such as the human and material
resources involved and analyzing the data needed for the training plan. In judging the
usefulness of the collected information, the validity of the overall training plan can be
examined, the improvement or elimination of the program explored, and the need for
the quality improvement of lecturers or educational administrators assessed. In addi-
tion, the evaluation validates the legitimacy of the educational investment and pro-
motes its usefulness. Thus, the significance of the training program evaluation can be
understood more clearly through the need for the evaluation.
In looking at the public sector, Sims (1993) explained that the evaluation of the
training program contributes to improving the effectiveness and efficiency of the pro-
gram, increasing the reliability of the activities of the training officers, providing
important information for the decision-making process, and increasing the commit-
ment level of decision makers regarding training. Accordingly, numerous training
institutes for public servants utilize various practices to assess the effectiveness of
their training program (Bjornberg et al., 2002). For instance, Broward County in
Florida used pre- and posttraining surveys targeting participants and their supervisors
to conduct three-level behavior evaluations. However, the method that continues to be
422
Public Personnel Management 46(4)
used most often in training program evaluations for public servants relies on the tradi-
tional Kirkpatrick’s four-level evaluation model; yet, this model only focuses on the
quantitative aspects and leaves out various other factors that could influence the effec-
tiveness of training.
The SCM
There is no doubt that Kirkpatrick’s model has made valuable contributions to training
evaluation theory and practice. This model poses the need for training evaluation
experts, intuitively presents the four results obtained through training, and simplifies
the complexities...
To continue reading
Request your trial