Analyzing the Effectiveness of Networks for Addressing Public Problems: Evidence from a Longitudinal Study

• Published date: 01 September 2021
• Author: Michael D. Siciliano,Jered B. Carr,Victor G. Hugg
• Date: 01 September 2021
• DOI: http://doi.org/10.1111/puar.13336

Analyzing the Effectiveness of Networks for Addressing Public Problems: Evidence from a Longitudinal Study 895

Research Article

Abstract: While scholars and practitioners frequently laud the potential of networks to address complex policy

problems, empirical evidence of the effectiveness of networks is scarce. This study examines how changes in network

structure (centralization and transitivity), network composition (sector diversity and geographic range), and tie

properties (stability and strength) influence community-level outcomes. Relying on a statutory requirement in the

state of Iowa requiring local governments to file all instances of intergovernmental and intersectoral collaboration, we

measure collaboration networks in 81 counties over 17 years in the areas of crime and economic development. Using

fixed effects models, we examine how changes in the structure and composition of these county-level networks affect

substantive policy outcomes. Our findings indicate that network properties matter, but that the specific properties may

be context dependent. We find network centralization and stability are stronger predictors of crime while network

composition is more strongly associated with economic development.

Evidence for Practice

• The performance of intergovernmental collaborations is rarely studied, either in terms of the outcomes of

individual agreements or from the networks resulting from multiple agreements.

• The properties (structure, composition, stability) of the networks that emerge from the actions of local

government officials to address critical public problems are generally not visible to the public, state policy

makers, or even the local officials themselves.

• Our research suggests that properties of these networks of agreements affect their impact on improving

public problems, suggesting opportunities for policy makers to encourage more effective structures.

• Our finding suggests that county networks effectively designed to improve economic development would

differ in meaningful ways from one effectively designed to reduce crime.

• Our findings indicate that network centralization and stability are stronger predictors of crime rates while

network composition is more strongly associated with indicators of economic development. Stable networks,

as measured by average amount of time the agreements are in place, may reduce violent crime, property

crime, and unemployment.

Despite the frequent use of networks and

corresponding growth in scholarly attention,

little is known about the structural forms

of networks most conducive to innovation and

performance (Turrini et al. 2010). This paper explores

how changes in the structure and composition of

county-level networks influence the effectiveness

of those networks in addressing public problems.

Understanding the relationship between network

characteristics and performance is critical as it can

help policy makers shape networks in ways most

conducive for success (Whetsell et al. 2020).

The success or performance of networks has been

measured at three levels: the community/clients

served by the network, the network itself, and the

individual actors or organizations within the network

(Provan and Milward 2001). Much of the literature

attempting to link networks to performance has

focused on the actor level (i.e., the individuals or

organizations which comprise the nodes in the

network) (e.g., Meier and O’Toole 2001, 2003;

Mewhirter and Berardo 2019). Research examining

network level performance emphasizes the network’s

internal governance, accountability mechanisms,

external legitimacy, and the range of services provided

(Carr, Blöschl, and Loucks 2012; Emerson and

Nabatchi 2015). While the performance of individual

actors and of the network itself, as an organizational

entity, is important, success at these levels does

not necessarily translate into effectively addressing

policy problems at the community level (Provan and

Milward 2001). Community-level outcomes can

be defined as the value provided, such as improved

Michael D. Siciliano

Jered B. Carr

Victor G. Hugg

University of Illinois at Chicago

Analyzing the Effectiveness of Networks for Addressing Public

Problems: Evidence from a Longitudinal Study

Victor G. Hugg received his PhD from the

Department of Public Administration at the

University of Illinois at Chicago in 2020. He

holds a Master’s Degree in political science

from the University of Illinois at Urbana-

Champaign. His research primarily focuses

on intergovernmental management and

collaborative governance, with an emphasis

on understanding network formation and

performance.

Email: hugg2@uic.edu

Jered B. Carr is a Professor and Head of

the Department of Public Administration,

and Co-Director of the Networks &

Governance Lab at the University of

Illinois at Chicago. Jered’s current research

focuses on the formation and performance

of urban governance networks, shared

public services/joint ventures, and the risk

perceptions of public officials considering

intergovernmental collaborations. He has

served as Co-Editor-in-Chief and Managing

Editor of the Urban Affairs Review since

2014.

Email: jbcarr@uic.edu

Michael D. Siciliano is an Associate

Professor in the Department of Public

Administration at the University of Illinois at

Chicago and Co-Director of the Networks

& Governance Lab. Michael studies how

humans and organizations collaborate

to improve society. His work explores the

cognitive, social, and institutional factors

influencing the formation and performance

of networks in the public sector. He currently

serves as associate editor of the

Journal of

Public Administration Research and Theory

.

Email: sicilian@uic.edu

Public Administration Review,

Vol. 81, Iss. 5, pp. 895–910. © 2020 by

The American Society for Public Administration.

DOI: 10.1111/puar.13336.

896 Public Administration Review • September | Octobe r 2021

access or the reduction in a problematic condition, to the area and

clients served by the network (Provan and Milward 2001). Because

public networks are used to address policy problems that spill

over organizational and jurisdictional boundaries, understanding

their implications at the collective or community level is essential

for targeting scarce resources toward effective service delivery

mechanisms (Provan and Milward 2001).

Unfortunately, little research exists to address how the structure and

composition of a network influence outcomes at the community

level. A systematic review of the network literature in 40 journals of

public administration and policy from 1997 to 2019 found only 21

empirical articles examining network effectiveness at the community

level (Medina et al. 2020). A major reason for this lack of research is

the significant data challenges facing statistical analyses of network

performance at the community level. Here, it is important to

make a distinction between studies that use the term network as a

form of organization and studies that use networks as a structural

phenomenon and collect data on the ties that link actors together.

Our interest in the effect of networks on community outcomes

is not simply a question about whether collaboration works, but

rather how the actual structure and composition of networks that

emerge through collaborative arrangements influence collective

performance. When measuring performance at the community

level, only a single observation of performance is available for each

network. In contrast, when looking at actor level performance, you

have an observation for every actor in the network. To statistically

model how network structures affect community-level performance,

data on a large number of networks are therefore required as each

network provides only a single observation. However, network data

are time-consuming to collect and require high response rates to

produce valid measurements (Borgatti, Everett, and Johnson 2013).

Consequently, prior empirical work examining the relationship

between network characteristics and performance outcomes relies

predominately on small sample methods (case studies, qualitative

comparative analysis) (Cristofoli, Macciò, and Pedrazzi 2015;

Provan and Milward 1995; Raab, Mannak, and Cambré 2015;

Sandström and Carlsson 2008; Wang 2016). In the few instances

where enough observations of complete networks have been

obtained to conduct statistical analysis using community-level

outcomes, the data have been cross-sectional (Kelman, Hong, and

Turbitt 2013; Nowell 2009; Yi 2018).

These studies, though reliant on small samples and cross-sectional

data, have produced important findings for our field. The body of

research suggests that community-level outcomes may be affected

by a network’s structure and composition. However, due to data

limitations, prior work has been (i) unable to control for time-

invariant unobservable factors that may influence performance

outcomes and (ii) unable to explore how, within a given network,

changes in structure and composition influence performance over

time. For example, the seminal work of Provan and Milward (1995)

found that centralized networks tend to perform better, and this

finding has been supported by other studies (Raab, Mannak, and

Cambré 2015). However, because prior work has relied on case

studies and cross-sectional data, it is unable to assess whether

increasing centralization leads to improved performance; it has

simply found an association between network centralization and

performance at a point in time. It could be that networks that are

more centralized result from having actors who are more powerful,

who possess more resources, and who are more willing to collaborate

with others, and it could be those aspects of the network rather than

its centralization, which are the drivers of performance.

We address these prior limitations by examining network

performance at the community level using a longitudinal dataset

of complete networks in the 81 counties in Iowa with populations

greater than 10,000. Our data on collaborations consist of

complete, whole network data on the agreements formed by local

governments in Iowa over a period of 17 years (2000–16). In the

forthcoming analysis, we assess how changes in the county-level

network structure and composition affect substantive performance

outcomes in two critical policy areas of crime and economic

development.

This article makes several contributions to our understanding

of network performance and local government collaboration.

Responding to Isett et al.’s (2011, i163) declaration that “[w]e face

significant limitations in our knowledge of how networks perform

over time,” this study statistically examines longitudinal data linking

networks and performance at the community level. Through these

longitudinal data, we show that changes in network structure

over time significantly influence community-level outcomes. This

suggests that while networks may be an effective means to address

collective challenges, the structure and evolution of the networks

(and not just the fact that actors are collaborating) are determinants

of collective outcomes. By exploiting an Iowa statutory requirement

that all intergovernmental or intersectoral agreements created by

public agencies be filed with the state government, this analysis

captures complete networks composed of both public and private

actors. In doing so, we are able to identify how the effect of network

composition change on performance is dependent on the policy area

in which the network operates. Finally, the article demonstrates how

emergent properties of networks formed predominantly through a

series of bilateral agreements have implications for the performance

of the broader policy system. Because the emergent characteristics

of networks matter, local, state, and federal government leaders can

work to develop incentives and systems that facilitate the formation

of networks in ways most amenable for addressing collective

problems (Whetsell et al. 2020).

Research Context: Polycentric Systems and Interlocal

Agreement Networks

The local government landscape in United States is characterized

by large numbers of relatively small governments that are

simultaneously interdependent yet substantially autonomous.

This “fragmentation” of political authority is seen by many to

simultaneously increase the problems facing governments and

undercut their ability to confront these challenges (Carr and

Siciliano 2019; Goodman 2019). Relying on small population

units to produce services creates an environment of service

production scale mismatch (Ostrom, Tiebout, and Warren 1961)

and exacerbates the problems of the disarticulated state

(Frederickson 1999). The fact that cities and counties, as general-

purpose governments, are responsible for providing many different

services to their residents guarantees that mismatches between

political boundaries and optimal production scale will exist. In

our study, we look particularly at the provision of services within