Strategic design under uncertain evaluations: structural analysis of design‐build auctions

• Published date: 01 September 2018
• Date: 01 September 2018
• Author: Hidenori Takahashi
• DOI: http://doi.org/10.1111/1756-2171.12246

RAND Journal of Economics

Vol.49, No. 3, Fall 2018

pp. 594–618

Strategic design under uncertain evaluations:

structural analysis of design-build auctions

Hidenori Takahashi∗

I investigate firms’ competition overprice and product design under uncertain design evaluations

in the context of Design-Build (DB) auctions. Reviewers’ design evaluations contain uncertainty

from a bidder’s perspective, leading luck to dampen differences in the firms’ chances of win-

ning. I model bidders’ behavior and show semiparametric identification of the model primitives.

Uncertain design evaluations increase the expected price of design quality and exacerbate an

auctioneer’s uncertainty in auction outcomes. These effects are mostly due to changes in bidding

strategies. Bid ranking swaps due to uncertain evaluations account for a small share of these

effects.

1. Introduction

Design competitions are widespread, ranging from grant applications in academic research

to the public design procurement of military weapons.1The outcomes of such design competitions

are, however, susceptible to a client’s subjective evaluation.Uncertain design evaluations introduce

an element of luck into design competitions: reducing the differences in the chances of winning

across designs of differing qualities.

I study the effect of a client’s uncertain evaluation on the suppliers’ design choices under

strategic interactions. A subjectivedesign evaluation may influence competition outcomes through

luck, which in turn affects the competing suppliers’ incentive to providea quality design. Despite

the fact that subjective evaluations are the norm in design competitions, their effect on suppliers’

behavior and competition outcomes are to date unknown.

To study suppliers’ behavior, I use hand-collected data on Design-Build (DB) auctions

from the Florida Department of Transportation (FDOT). DB auctions are used not only by state

∗University of Mannheim; htakahas@mail.uni-mannheim.de.

This article is based on Chapter 1 of my dissertation, completed in the Department of Economics at University of Toronto.

I am indebted to Victor Aguirregabiria for invaluable advice and generous guidance. I would also like to thank Andre

Boik, Ettore Damiano, Francesco Decarolis, Isis Durrmeyer, Rahul Deb, Serafin J. Grundl, Takakazu Honryo, Yao Luo,

Robert McMillan, Xianwen Shi, Michelle Sovinsky,Junichi Suzuki, Yuya Takahashi, Naoki Wakamori, Yuanyuan Wan,

and the participants in CEPR/JIE (Bologna, 2013), EARIE (´

Evora, 2013), and Jornadas de Econom´

ıa Industrial (Segovia,

2013) for helpful comments. Thanks to Daniel Garcia and Andrea Pozzi for extensivediscussions. Suggestions made by

the Editor and two anonymous referees significantly improvedthe article. All errors are my own.

1Public Private Partnerships, which havesurged in popularity among practitioners, are also an example of public

procurement that involves a design competition among consulting firms.

594 C2018, The RAND Corporation.

TAKAHASHI / 595

departments of transportation in the United States, but also in many other countries.2The US

Department of Defense extensively uses DB auctions to procure military construction projects

and military weapons, which together cost taxpayers billions of dollars annually.3

In a DB auction, bidders compete over price and design to win a contract to deliver an

infrastructure project, ranging from bridge repairs to building construction. Upon receiving price

and design proposals, each reviewer of the FDOTindependently evaluates and assigns a score to

every design proposal. The average across the reviewers’ evaluations then determines the quality

score of a design proposal. The bidder with the lowest price per quality score ratio (PQR) wins

the project and receives its price bid upon completing the project.4

The data reveal a substantial degree of discrepancy among reviewers for a given design

proposal, and conversations with contractors confirm that uncertain design evaluations are of

significant concern to them. Such uncertainty in design evaluations, which I refer to as evaluation

uncertainty, has not been considered to date in the vast auction literature. Indeed, to the best

of my knowledge, there is virtually no empirical work that has investigated the implications of

uncertain design evaluations on supplier behavior.

To guide the analysis, I develop a model in which each bidder strategically chooses its price

and design quality in the face of uncertain design evaluations. The model allows for complex

bidding strategies through multidimensional types: bidders are heterogeneous in the variable cost

of providing a quality design and in the fixed cost of implementing the designed project. The

introduction of a nonlinear quality cost function also allows for a flexible substitution pattern

between price and design quality choices.

A large degree of uncertainty in design evaluation implies that project allocation is heavily

influenced by luck. The effect of luck on project allocation is especially pronounced when com-

peting designs are comparable in quality. These designs are perfectly ranked without evaluation

noise but face an equal chance of winning with an infinite amount of evaluation uncertainty. I

find that the reduction in the differences in the bidders’ chances of winning results in: (i) a higher

expected price per unit of design quality; and (ii) a greater spread in price and design quality.That

is, greater evaluation uncertainty worsens the expected price of design quality, and exacerbates

the uncertainty in auction outcomes from the auctioneer’s point of view.

Identification of the model is challenging. The econometrician does not observe bidders’

design quality choices but instead observes some noisy evaluations of designs, implying that the

distribution of bidders’ costs needs to be identified separately from the distribution of noisy quality

signals. The problem of unobserved quality choices is further complicated by the endogeneity of

design quality choices. As the bidders’ design quality choices depend on the bidders’ unobserved

costs of producing quality designs, the identification of the quality cost function requires some

cost-irrelevant variable.

Unobserved heterogeneities further exacerbate the identification problem. Procurement auc-

tions of infrastructure projects are known to contain a significant degree of unobserved auc-

tion heterogeneity: the cost commonly shared and observed by bidders but unobserved by the

econometrician. Ignoring unobserved auction heterogeneity exaggerates the extent of bidder

heterogeneities, which in turn undermines the effect of uncertain design evaluations on bidding

incentives.The deg ree of evaluationuncertainty could also be confounded by unobserved reviewer

2As of October 2010, there are 39 state departments of transportation that use DB, including California, Delaware,

Georgia, and Minnesota. DB auctions are also common in other developed countries, including Canada, Italy, Japan, and

Sweden.

3The National Defense Authorization Act for the Fiscal Year 2015 (NDAA) bill, which became public law

in December 2014, explicitly prohibits the use of price-only auctions to procure construction services for military

construction contracts. The Associated General Contractors of America, which consists of 26,000 construction firms,

expresses its support to ban the use of price-only auctions for construction services for the reason that price-only auctions

ignore many aspects of design quality.

4PQR is a winner selection rule used by many state departments of transportation, including Alaska, Michigan,

North Carolina, and South Dakota.

C

The RAND Corporation 2018.