Markets for ideas: prize structure, entry limits, and the design of ideation contests

• Published date: 01 June 2020
• Date: 01 June 2020
• Author: Pavel Kireyev
• DOI: http://doi.org/10.1111/1756-2171.12325

RAND Journal of Economics

Vol.51, No. 2, Summer 2020

pp. 563–588

Markets for ideas: prize structure, entry

limits, and the design of ideation contests

Pavel Kireyev∗

I develop an empirical model of idea generation contests with heterogeneous participants and

endogenous entry, fit the model to data from a platform used by major advertisers, and simulate

counterfactual contest designs. The empirical model resolves ambiguous predictions yielded by

contest theory about the effects of different prize structures on contest outcomes. Simulations

reveal the impact of strategies that hold fixed total award and balance competition by handicap-

ping advantaged participants. Increasing the number of prizes while restricting the number of

prizes per participant can improve outcomes for the platform. The results provide guidance for

the design of large contests.

1. Introduction

Contests have a rich history as a mechanism for the procurement of innovation. Firms in-

volved in research and development use contests to procure ideas for newtechnologies, products,

and even solutions to scientific problems. Firms in creative industries, such as advertising, de-

sign, and architecture, extensively rely on contests and pitch competitions to identify the most

promising ideas. Recently, firms have turned to online innovation platforms to attract a large

number of submissions and improve the efficiency of their procurement efforts. For example,

Colgate-Palmolive, a large consumer products firm, turned to a digital creative agency to gen-

erate ideas for one of its most visible advertising campaigns. Colgate-Palmolive organized an

online ideation contest—members of the agency’s innovation platform submitted ad ideas for

a chance to win a cash prize. The winning ideas were used to develop an ad that aired during

a $4 million spot shown to over 109 million viewers of North America’s most popular yearly

television event—the Super Bowl.

Advertisers are not alone in adopting a scalable model of ideation. Government agencies

and firms across a variety of industries have implemented ideation contests. For example, Chal-

lenge.gov, a government-operated innovation platform, solicits ideas for projects organized by

federal agencies such as DARPA and NASA. Innocentive, a platform for scientific innovation,

∗INSEAD Europe Campus, France; pavel.kireyev@insead.edu.

I thank my committee, Elie Ofek, Sunil Gupta, and Ariel Pakes for their support. I thank Karim Lakhani, Robin Lee,

Donald Ngwe, Daniel Pollmann, Al Silk, Thomas Wollmann, and participants at numerous seminars for helpful discus-

sions. I thank the editor and two anonymous referees for their helpful comments. I thank the data providerfor providing

insight into the data and sharing useful information about innovation platforms. All errors are my own.

© 2020, The RAND Corporation. 563

564 / THE RAND JOURNAL OF ECONOMICS

hosts ideation contests for companies such as Ford, GlaxoSmithKline, and MasterCard. Uni-

versities, large corporations, and startup incubators organize ideation contests using proprietary

platforms to collect, score, and reward the best ideas.

Contests may attract a large number of participants who differ considerably in their poten-

tial. Although theories abound on how to best design contests with heterogeneous participants

(Moldovanu and Sela, 2001; Szymanski and Valletti, 2005; Terwiesch and Xu, 2008), the

directional impact of a design decision often depends on the extent of participant heterogeneity.

As a result, theory often yields ambiguous predictions about the impact of a design decision. Few

articles have attempted to quantify heterogeneity or the impact of different design parameters

in a field setting. In this research, I develop an empirical framework for answering the following

question: How can a sponsor design a contest with heterogeneous participants to achieve the

desired participation or quality outcomes? This empirical framework can be applied to data on

contest participation and victories in large contests to identify the extent of participant hetero-

geneity and connect it to optimal design decisions. Thereby, I am able to resolve the ambiguous

predictions of theoretical models by relating their inputs directly to data. I examine the impact

of several important design decisions that hold fixed total award—how many prizes to award per

contest, how many prizes to allow per participant within a contest, and howmany submissions to

accept per participant—on entry, submissions, and idea quality outcomes such as expected total

quality.

I develop and estimate a structural model of ideation contests using data from a popular

innovation platform. The model captures participant, jury, and sponsor decision processes. Par-

ticipants choose how many ideas to submit to a contest based on their expected returns and costs

of effort. A jury assigns a quality rating to all submissions. The sponsor then ranks submissions

and rewards the winners. Contests are characterized by a large number of possible design de-

cisions. A structural model allows the researcher to vary different levers, identify those that are

most important, and guide the platform’schoice of which designs to implement in future contests

and which experiments to run to explore the effects of previously unaltered design levers. The

rich contest theory literature can inform model assumptions and make transparent the connection

between observed participant behavior, the predicted impact of a design decision, and the under-

lying theoretical mechanism. In addition, the space of possible design decisions for contests with

many participants is very large—it involves all possible combinations of prizes for any number

of winners, potentially coupled with restrictions on entry. A descriptive analysis of the data may

only identify designs which performed well among the set of attempted designs, which is usu-

ally very small compared to the set of possibilities. A structural model provides a theoretical

underpinning for how certain classes of designs affect participant behavior and is able to identify

strategies which may be effective but have not been previously attempted. Indeed, through coun-

terfactual simulations, I find that sponsors would have been better off had they offered a larger

number of prizes and restricted the maximum number of prizes per participant.

I estimate the structural model in three stages. First, data on sponsor rankings of winning

submissions and the ratings assigned to all submissions by a jury identify the chance that a sub-

mission will win given a particular rating. Second, jury ratings, participant characteristics, and

the availability of multiple submission per participant identify the distribution of ratings condi-

tional on observable and unobservable participant characteristics. Also, I allow for jury ratings

to depend on idea characteristics such as timing, length, and sentiment which I recover using nat-

ural language processing techniques. Third, participant submission decisions identify the costs

of ideation. I estimate the final stage as an empirical discrete game where participants choose

how many ideas to submit to a given contest to maximize their expected payoffs and may choose

not to submit any ideas at all. I use moment inequalities to partially identify parameters of the

cost function. This methodology allows for multiple equilibria, a nonparametric cost unobserv-

able, and yields estimates that are robust to simulation error, optimization mistakes, and different

specifications of participant information sets.

C

The RAND Corporation 2020.