Logistics Information System Evaluation: Assessing External Technology Integration and Supporting Organizational Learning

• Published date: 01 December 2014
• Author: John E. Bell,Randy V. Bradley,Brian S. Fugate,Benjamin T. Hazen
• Date: 01 December 2014
• DOI: http://doi.org/10.1111/jbl.12075

Logistics Information System Evaluation: Assessing External

Technology Integration and Supporting Organizational Learning

John E. Bell

1

, Randy V. Bradley

1

, Brian S. Fugate

2

, and Benjamin T. Hazen

1

1

The University of Tennessee

2

Colorado State University

Firms often outsource the development and acquisition of logistics information systems (LIS) needed to improve logistics processes. Manag-

ers tasked with such outsourcing decisions often struggle to understand and balance the external technology’s impact on existing logistics

processes, individual stakeholders, firm strategies, and the financial and operational performance of the firm. Unfortunately, research is limited

on (1) methods for evaluating the performance capabilities of systems from external sources prior to full implementation, and (2) the impact of

external technology integration (ETI) on organizational behavior and learning related to a firm’s logistics processes. Through the lens of organi-

zational learning (OL) theory, this research uses a case study approach to examine the transportation division of a major U.S.-based fuel retailer

to gain insights on the management control of ETI efforts. The study builds theory to fill important literature gaps then develops a conceptual

framework and supporting propositions to inform future research on logistics ETI. The findings highlight important OL implications for firms

involved in ETI efforts and also provide a practically relevant management control tool that can be used by logistics practitioners.

Keywords: external technology integration; information systems success; logistics information systems; organizational learning; vehicle routing.

INTRODUCTION

In the last several decades, automating previously manual pro-

cesses via information systems (IS) has been at the heart of

improved logistics productivity (Byrd and Davidson 2003; Rai

et al. 2006; Barratt and Oke 2007). While many firms have

proven that successfully leveraging IS can improve logistics per-

formance and yield substantial cost savings and competitive

advantage, other firms fail to derive the expected level of strate-

gic value from their IS investments (Richey et al. 2009; Bradley

et al. 2011; Fawcett et al. 2011). Poor IS adoption decisions risk

erosion of competitiveness, failure to meet business goals, and

loss of shareholder value (Favilla and Fearne 2005). When firms

undertake an external technology integration (ETI) effort, that is,

the acquisition and incorporation of technology from an external

source, the risks can be exacerbated (Stock and Tatikonda 2000,

2008). Thus, a sound management control system is needed to

evaluate the potential success and multilevel outcomes of a logis-

tics ETI effort, especially when the level of risk is high (Long

et al. 2010).

This study evaluates a logistics ETI effort using a manage-

ment control system derived from DeLone and McLean’s

(1992, 2003) information systems success (ISS) model.

Although the ISS model has been extensively used in the IS

literature as the basis to evaluate the postimplementation impact

(i.e., success) of IS adoption, previous studies make no distinc-

tion between the evaluation of systems acquired from external

sources and systems developed internally. However, Stock and

Tatikonda (2000) clearly articulate performance differences

resulting from different levels of technology uncertainty and

organizational interaction when comparing an ETI with an

internally developed technology. Unfortunately, the current liter-

ature and theory pertaining to ETI is virtually silent on when

and how to implement management control mechanisms that

evaluate implementation outcomes. In addition, literature regard-

ing the organizational learning (OL) implications of such man-

agement control efforts, and the impact of such OL on the final

evaluation of ETI project success is also lacking. This knowl-

edge gap is especially disappointing to those interested in

implementing external logistics information systems (LIS),

which tend to span several inter- and intraorganizational bound-

aries and, thus, typically demand a greater degree of resources

and management support for implementation (Bardi et al.

1994). Herein, we consider LIS to include transportation man-

agement systems, vehicle routing/scheduling systems, warehouse

management systems, enterprise resource planning systems, and

other IS that support logistics operations.

The purpose of this study is twofold. From a theory-building

perspective, we construct a conceptual model that elucidates

understanding of the outcomes of ETI efforts and offers research

propositions to guide future research. From a practitioner per-

spective, we develop and validate a management control method

that includes an LIS evaluation framework to help guide firms

when making ETI decisions in support of logistics operations,

and to examine the outcomes of using such a method. The

research presented herein follows an iterative case study

approach and works closely with FuelCo (a pseudonym), a major

U.S.-based fuel retailer. An LIS evaluation framework, which is

based on the ISS model, was codeveloped with FuelCo over a

single year from 2010 to 2011. This newly developed LIS evalu-

ation framework was used by the company during the case study

period as part of the potential acquisition of a vehicle routing

software application from an external technology source. The

LIS evaluation framework was used to measure the ETI effort’s

potential for success and to decide whether to implement the LIS

throughout the corporation. The results of the exploratory study

Corresponding author:

Brian S. Fugate, Department of Management, Colorado State Uni-

versity, 223 Rockwell Hall, Ft. Collins, CO 80523-1275, USA;

E-mail: brian.fugate@colostate.edu

All authors contributed equally and, thus, are listed alphabetically.

Journal of Business Logistics, 2014, 35(4): 338–358 doi: 10.1111/jbl.12075

© Council of Supply Chain Management Professionals

provide theoretical insights about when and how to manage ETI

efforts. Additionally, the case study results help identify how and

where OL can occur within a firm and its impact on overall pro-

ject success, regardless of the final ETI adoption decision. Fur-

thermore, the case results contain implications on how to

improve performance measurement criteria for future ETI efforts.

The remainder of this article is structured as follows. This arti-

cle first considers the ETI, ISS, and OL literatures, which pro-

vide the theoretical foundation for this study and highlights

critical knowledge gaps. Next, the methodology and data collec-

tion are introduced. Subsequently, after a discussion of the

results and their broader implications for logistics research and

practice, the manuscript concludes with research limitations and

potential future research.

THEORETICAL FOUNDATION

Research on ETI suggests that ETI performance is dependent on

the “fit”between the level of uncertainty of the technology

(information processing requirements) and the level of interorga-

nizational interaction between the supplying and purchasing firm

(information processing capabilities) (Stock and Tatikonda 2000,

2004, 2008; Tatikonda and Stock 2003). The results of the afore-

mentioned research suggest high levels of technology uncertainty

create a need for high levels of interorganizational interaction to

reduce equivocality. This leads to greater ETI performance (i.e.,

effectiveness). Stock and Tatikonda (2008) specify three subdi-

mensions for measuring ETI project performance: the time

required to complete the ETI project, the ETI-related costs, and

the functional operation/technical performance. Additionally, they

provide insight into overall performance based on the project

criticality, user participation, and the purchasing firm’s ETI expe-

rience.

Stock and Tatikonda’s (2008) research provides a theoretical

link between ETI and performance, yet their work is limited in

two areas that have important implications for theory and prac-

tice of ETI. First, the authors acknowledge that the performance

measures used in their study “are traditional tactically oriented

project objectives. There are other dimensions of project perfor-

mance such as learning,user satisfaction with the technology,

and strategic competitive advantage gained from integration of

the technology”(Stock and Tatikonda 2008, 78).

Indeed, firms often turn to external providers for systems that

will provide operational, tactical, and strategic value and benefits

(Young et al. 2009; Hazen and Byrd 2012; Jandhyala 2012).

Operational-level value is derived from improvements in the exe-

cution of day-to-day operations by way of better coordination of

activities and processes necessary for the efficient and effective

fulfillment of customer orders. Tactical value is derived from

improvement of allocating human resources (e.g., dispatchers and

drivers), physical resources (e.g., vehicles), administrative effi-

ciencies and productivity, and improvement in quality of cus-

tomer service rendered. An added distinction between tactical

and operational relates to the frequency of decisions and the

longevity of their ensuing benefits. As such, operational pertains

to more day-to-day operations and decisions whose instantiation

and impact are more relatively immediate and short-lived. Tacti-

cal pertains to operations and decisions that happen less

frequently (e.g., weekly demand forecasts and distribution and

transportation planning) and whose instantiation can take weeks

or months and impacts are of a medium duration. Much confu-

sion around the difference between tactical and operational stems

from the elusive understanding that operational activities, pro-

cesses, and decisions are often tied to and, in some cases, in

response to changes to environmental factors that confound or

complicate prior tactical decisions. Stock and Tatikonda (2008)

admit, however, that research is limited on ETI’s strategic value

to the organization, which pertains to benefits and capabilities

that will result in the creation of competitive barriers and compe-

tencies and/or resources that cannot easily be replicated by com-

petitors.

Second, Stock and Tatikonda (2008) call for more robust mea-

sures of ETI effectiveness. They state that their measures allow for

the implicit weighting of different performance dimensions. This

approach can lead to more subjectivity in an evaluation process

that is already riddled with complexities, thereby creating the need

for a more objective approach to measuring ETI effectiveness.

Accordingly, the current study draws from OL theory to

explore the outcomes of ETI efforts. Importantly, inherent in OL

theory is the contention that success involves more than initial

project (e.g., ETI) performance and that robust evaluation pro-

cesses (management control systems) are necessary to enable

project success factors that are more strategic in nature.

OL and ETI

OL is the process by which organizations as collectives learn

through interaction with their environments (Cyert and March

1963), and has been defined as the dynamic process of strategic

renewal (Bontis et al. 2002). Thus, the ultimate outcome of

effective OL is the strategic renewal of the organization, which

is defined as “an evolutionary process associated with promoting,

accommodating, and utilizing new knowledge and innovative

behavior to bring about change in an organization’s core compe-

tencies”(Floyd and Lane 2000, 155). OL enables ongoing strate-

gic renewal through processes that overcome strategic inertia,

which is the common tendency for organizations to remain with

the status quo (Hannan and Freeman 1984).

At its most basic level, OL theory is broadly based on two

premises (Bontis et al. 2002): First, OL involves both assimilat-

ing new learning (exploration) and using what has already been

learned (exploitation) through feed-forward and feedback flows.

Second, OL is multi-level—individual, group, and organizational

(Argyris and Sch€

on 1978). Crossan et al. (1999) recognize the

need to combine the two premises of OL theory and conse-

quently developed the seminal 4I framework. This framework

(seen in Figure 1) links the feed-forward and feedback flows

across the individual, group, and organizational levels by four

broad categories of social and psychological micro-processes:

intuiting, interpreting, integrating, and institutionalizing. Intuiting

is the beginning process of OL that occurs at the individual level

and involves preconscious recognition of past patterns and/or

future possibilities. Interpreting is the conscious process of learn-

ing in which individuals develop cognitive maps about their lan-

guage and communication within a group in which they operate.

Integrating is the process of developing shared understanding

among individuals through dialogue and joint action. Institution-

Logistics Information System Evaluation 339