Supply chain management research: Key elements of study design and statistical testing

• Author: Catherine A. Helmuth,Donovan Y. Collier,Joe B. Hanna,Christopher W. Craighead,Brian L. Connelly
• Date: 01 May 2015
• DOI: http://doi.org/10.1016/j.jom.2014.12.001
• Published date: 01 May 2015

Journal of Operations Management 36 (2015) 178–186

Contents lists available at ScienceDirect

Journal of Operations Management

journal homepage: www.elsevier.com/locate/jom

Technical note

Supply chain management research: Key elements of study design

and statistical testing

Catherine A. Helmutha, Christopher W. Craighead b, Brian L. Connellya,∗,

Donovan Y. Colliera, Joe B. Hanna a

aRaymond J. Harbert College of Business, Auburn University, 415 West Magnolia Avenue, Auburn, AL 36849, United States

bSmeal College of Business, The Penn State University, Business Building, Suite 483, University Park, PA 16802, United States

article info

Article history:

Received 19 July 2013

Received in revised form 3 November 2014

Accepted 14 December 2014

Available online 22 December 2014

Accepted by Thomas Younghoon Choi

Keywords:

Supply chain

Effect size

Statistical power

Reliability

Empirical research

abstract

Over the past three decades, supply chain management (SCM) has evolved from its origins as a nascent

field of study to encompass construct definition, identification of the field’s central issues, and establish-

ment of its conceptual boundaries. At this point, a sufficient body of empirical SCM research has been put

forward to allow for quantitative assessment of the field. Therefore, we examine three key elements of

study design to assess what has happened, what is currently happening, and where we should be heading

as a field. To do so, following a pattern of reviews in similar disciplines, we begin with an examination

of effect sizes of the relationships under investigation. Results show that effect sizes in SCM research

have marginally increased over time and that sub-domains within SCM that receive the most scholarly

attention also have higher effect sizes. We also conduct a post hoc analysis of statistical power and empir-

ically examine a range of factors and study contexts that could influence power. Findings suggest that

average statistical power in SCM research exceeds the statistical power of most related disciplines and is

particularly high in several unique contexts. Lastly, we find that measurement reliability and the use of

control variables have increased over time, possibly suggesting the field has matured, instilling a degree

of confidence in its research. Overall, our results show that SCM research is becoming more empirically

rigorous, but we also uncover key areas that warrant improvement. We describe implications of our

review for the design of future SCM empirical studies.

© 2014 Elsevier B.V. All rights reserved.

1. Introduction

A “supply chain” refers to the activities, functions and entities

that are connected via product and information flow from source to

consumer (Craighead et al., 2007). Research on supply chain man-

agement (SCM) has evolved from its origins as a nascent field of

study to encompass construct definition, identification of the field’s

central issues, and establishment of its conceptual boundaries. We

believe a sufficient body of empirical work has emerged (Craighead

and Meredith, 2008), particularly over the past ten years, to warrant

more quantitative assessment of study designs. Reviews of specific

topic areas and theories are useful for summarizing content (Short,

2009), but it is also important to examine and assess a disciplines’

methodological rigor. Such assessments have been instrumental

in the social sciences and important to assessing study design in

fields such as entrepreneurship (Connelly et al., 2010), manage-

ment (Cashen and Geiger, 2004), and industrial and organizational

∗Corresponding author. Tel.: +1 334 703 7070.

E-mail address: bconnelly@auburn.edu (B.L. Connelly).

psychology (Mone et al., 1996). Therefore, in this technical note, we

examine three core aspects of SCM study design and consider what

has happened, what is currently happening, and where we should

be heading as a field of study.

To evaluate study design of empirical SCM research, the most

basic question is: what are researchers trying to measure? This

speaks to the issue of effect size, which describes the strength

of association between two variables, a predictor and criterion

(Cohen et al., 2003) (other reviews answer this question from

a theoretical standpoint, examining the range of research ques-

tions and theories that SCM researchers employ). Effect size is

important because empirical SCM research is largely built around

statistical inference testing, so we should begin by examining

what it is researchers are testing. That is, whether it is some-

thing that is actually occurring or not. Effect size captures the

extent to which the theoretical phenomenon that the researcher

has chosen to examine actually exists in the population (Cohen,

1988).

Assuming the presence of an effect for the relationships under

investigation, we also consider whether researchers are exam-

ining a sufficient pool in order to draw conclusions about that

http://dx.doi.org/10.1016/j.jom.2014.12.001

0272-6963/© 2014 Elsevier B.V. All rights reserved.