When do in‐service teacher training and books improve student achievement? Experimental evidence from Mongolia

• Date: 01 August 2018
• Author: Habtamu Fuje,Prateek Tandon
• Published date: 01 August 2018
• DOI: http://doi.org/10.1111/rode.12387

REGULAR ARTICLE

When do in-service teacher training and books

improve student achievement? Experimental

evidence from Mongolia

Habtamu Fuje

|

Prateek Tandon

The World Bank

Correspondence

Habtamu Fuje, The World Bank,

Washington, DC 20433, USA.

Email: Habtamu.Fuje@columbia.edu or

HFuje@worldbank.org

Abstract

This study presents evidence from a randomized controlled

trial in Mongolia on the impact of in-service teacher train-

ing and books, both as separate educational inputs and as a

package. It tests for complementarity of inputs and nonlin-

earity of returns from education investment, as measured

in students’test scores. The result suggests that the provi-

sion of books, on top of teacher training, raises students’

achievement substantially. However, teacher training and

books weakly improve test scores when provided individu-

ally. This study sheds light on the relevance of supple-

menting teacher training schemes with appropriate

teaching materials in resource-poor settings.

1

|

INTRODUCTION

Policy-makers and practitioners in developing and developed countries often invest heavily in brief

in-service teacher training to enhance education outcomes. Spurred by the Millennium Develop-

ment Goals (MDGs), developing countries have also rapidly expanded school infrastructure in the

past decade and ramped up in-service teacher training. These investments have aimed to satisfy the

growing demand for teachers and help improve educational quality (Bunyi et al., 2013; Govern-

ment of Mongolia, 2007; Kidwai et al., 2013). However, conclusive evidence on the impact of in-

service teacher training on student achievement—as measured by a comprehensive set of test

scores—scarcely exists, particularly in developing countries. Moreover, the differential impact of

such training on achievement when students and teachers have access to appropriate books to

effectively implement the lessons learned during training—versus when they do not—has not been

investigated. Previous studies have focused on the individual provision of either teacher training or

books and have not examined the potential complementarity between these inputs.

©2018 The World Bank Review of Development Economics ©2018 John Wiley & Sons Ltd

DOI: 10.1111/rode.12387

1360

|

wileyonlinelibrary.com/journal/rode Rev Dev Econ. 2018;22:1360–1383.

Properly documenting the impact of such investments on student outcomes can address this

gap. The few rigorous evaluations of teacher training programs conducted to date suggest a

moderate potential to improve student outcomes, but the evidence is mixed. A recent system-

atic review by Glewwe et al. (2013), which examined impact evaluation studies from the per-

iod 1990–2010, concluded that there is only modest evidence that teacher training improves

student test scores. Specifically, 11 of the 29 estimates included in their analysis demonstrate

positive, significant impacts (one is significant and negative). But only three of these studies

were well identified, experimental, or based on natural experiments. Other works on the

impacts of teacher training also do not provide conclusive positive evidence: improvements in

test scores were documented by some (see Jacob and Lefgren, 2004; Raudenbush et al., 1993;

Zhang et al., 2013), while others find no evidence (see Angrist and Lavy, 2001; Harris and

Sass, 2011; Lai et al., 2011). Evans and Popova (2014) noted that the type of teacher training

and the specific details of the intervention and the settings also matter. They argue that varia-

tion in outcomes (as reported in systematic reviews/meta-analyses) for the same class of inter-

vention is due to aggregation and ignoring the specific circumstance and details of

interventions. In Mongolia, the current study explores the effectiveness of teacher training when

it is complemented by books to facilitate learning.

With regard to the impacts of books, the same review by Glewwe et al. (2013) revealed that, in

general, there is strong, but non-unanimous, evidence for the positive impact of textbooks and

workbooks on student learning. However, when considering well-identified studies only, they

noted weak evidence. Older studies suggest that books improve achievement (Heyneman et al.,

1984; Jamison et al., 1981), while more recent studies in Kenya (Glew we et al., 1998, 2009) and

in Sierra Leone (Sabarwal et al., 2014) contradict these findings.

Most of these previous studies, however, have had some methodological limitations. The

most serious methodological issue with observational studies is the non-random assignment of

teachers to in-service training programs or students to book provision. A few quasi-experi-

mental studies have attempted to address these issues (Angrist and Lavy, 2001; Jacob and

Lefgren, 2004; Rothstein, 2010). A number of issues arising from non-random assignment

need to be addressed. For instance, factors like self-initiation, relationships with supervisors,

personal connections and political participation confoundwithateacher’s decision to attend

in-service training as well as her general motivation and capacity to teach (see Jacob and

Lefgren, 2004). Similarly, a student’s access to books confounds with a number of other

covariates such as parental education, wealth, and school resources, which directly affect stu-

dent outcomes.

This study uses data from the randomized assignment of teachers into a training program or the

provision of books to randomly selected primary schools in Mongolia under the Rural Education

and Development (READ) project to examine the impacts of these interventions on student

achievement. In addition, due to the changes in the initial experiment design, propensity score

matching is used (see Section 3.2). The randomization is nationally representative—it covers the

entire rural population of the whole country, as opposed to a typical small-scale randomization

study from which generalization to the national population is not feasible. This enables us to

address limitations arising from non-random assignment and provides a basis for generalizing

about the impact of the interventions.

In addition, this study investigates the differential impact of in-service teacher training or book

provision as a stand-alone intervention vis-

a-vis in-service training accompanied by provision of

age-appropriate books. Some previous evidence on the topic suggests that provision of education

inputs as a bundle is more effective in improving outcomes (see Conn, 2014; Evans and Popova,

FUJE AND TANDON

|

1361