Product variety, across‐market demand heterogeneity, and the value of online retail

• Published date: 01 December 2018
• Author: Thomas W. Quan,Kevin R. Williams
• DOI: http://doi.org/10.1111/1756-2171.12255
• Date: 01 December 2018

RAND Journal of Economics

Vol.49, No. 4, Winter 2018

pp. 877–913

Product variety, across-market demand

heterogeneity, and the value of online retail

Thomas W. Quan∗

and

Kevin R. Williams∗∗

Online retail gives consumers access to an astonishing variety of products. However, the addi-

tional value created by this variety depends on the extent to which local retailers already satisfy

local demand. To quantify the gains and account for local demand, we use detailed data from

an online retailer and propose methodology to address a common issue in such data—sparsity

of local sales due to sampling and a significant number of local zeros. Our estimates indicate

products face substantial demand heterogeneity across markets; as a result, we find gains from

online variety that are 45% lower than previous studies.

1. Introduction

There is widespread recognition that as economies advance, consumers benefit from in-

creasing access to variety. Several strands of the economics literature have examined the value

of new products and increases in variety either theoretically or empirically, for example, in trade

(Krugman, 1979; Broda and Weinstein, 2006; Arkolakis et al., 2008), macroeconomics (Romer,

1994), and industrial organization (Lancaster, 1966; Dixit and Stiglitz, 1977; Brynjolfsson, Hu,

and Smith, 2003). The Internet has given consumers access to an astonishing level of variety.

∗University of Georgia; quan@uga.edu.

∗∗Yale University; kevin.williams@yale.edu.

We are indebted to Thomas Holmes, Amil Petrin, Kyoo il Kim, and Joel Waldfogel for all of their helpful advice. We

would also like to thank the Editor, Marc Rysman, two anonymous referees, Pat Bajari, Steve Berry, Tat Chan, Judy

Chevalier, Ying Fan, Fiona Scott Morton, Ben Shiller, Catherine Tucker, and especially Chris Conlon, Amit Gandhi,

Aniko ¨

Ory, and MykhayloShkolnikov for useful comments. We also thank the participants at numerous conferences and

seminars. In 2015: Darmouth-Tuck Winter IO, Northeastern University, Washington University at St. Louis, Michigan

State University,Department of Justice, Bureau of Labor Statistics, Federal Communications Commission, University of

Georgia, Universityof California at Los Angeles, NBER Winter Digitization Meeting (Palo Alto), Singapore Management

University,National University of Singapore, Stanford University (Marketing and Economics), IIOC (Boston), University

of Maryland, University of Toronto, NYU Stern IO Day, QME (Cambridge, MA). In 2016: Harvard University, MIT,

Boston College,NBER IO Summer Institute (Cambridge, MA). In 2017: the Federal Reserve Bank of Minneapolis. Finally,

we thank the Minnesota Supercomputing Institute (MSI) for providingcomputational resources, Mauricio C ´

aceres Bravo

for his excellent research assistance, and the online retailer for allowing us to collect the data used in this study.

C2018, The RAND Corporation. 877

878 / THE RAND JOURNAL OF ECONOMICS

Consider shoe retail. A large traditional brick-and-mortar shoe retailer offers at most a few thou-

sand distinct varieties of shoes. However, as we will see, an online retailer may offer over 50,000

distinct varieties. How does such a dramatic increase in product variety contribute to welfare?

The central idea of this article is that the gains from online variety depend critically on

the extent to which demand varies across geographies and on how traditional brick-and-mortar

retailers respond to those local tastes (Waldfogel, 2008, 2010; Thomas, 2011).1For example,

online access to an additional 5000 different kinds of winter boots will be of little value to

consumers living in Florida, just as access to an additional 5000 different kinds of sandals will be

of little consequence to consumers in Alaska. If Alaskan retailers already offer a large selection

of boots that captures the majority of local demand, only consumers with niche tastes—possibly

those who have a trip planned to Florida—will benefit from the additional variety offered by

online retail. Therefore, in order to quantify the gains from variety due to online retail, it is

critical to estimate the extent to which demand varies both within and across locations.

Influential work by Brynjolfsson, Hu, and Smith (2003) found significant gains to consumer

welfare ($731 million–$1.03 billion in 2000) due to the increase in access to book varieties

provided by Amazon.com. They estimate the gain to consumers from increased variety to be 7

to 10 times larger than the gain derived from the competitive price effect. These gains have since

been dubbed the “long tail” benefit of online retail by Anderson (2004). This term captures the

idea that although the value of each additional variety individuallycontributes only a tiny fraction

to consumer welfare, the summation of an enormous number of small gains becomes sizable.

However, the existing literature has been limited by the aggregate nature of their data and have

been forced to abstract from local heterogeneity.

In this article, we revisit the gains from variety made available by e-commerce for a com-

monly purchased good, shoes. We begin by collecting new data containing millions of geograph-

ically disaggregated footwear sales, daily inventory, and all product reviews from a large online

retailer. We present descriptive evidence that heterogeneity in consumer tastes across markets

is substantial. As a result, we would expect that local retailers respond to these differences in

local demand across locations and show supporting evidence of this using supplementary brick-

and-mortar assortment data. We then employ a structural model of demand to obtain empirical

estimates of the value of increased variety due to online retail. Our results show that omitting the

role of local tastes and retailer responses leads to estimates that significantly overstate the gains

from online variety. In our application, we find this overstatement to be over 75%.

Our results have two major policyimplications. First, the disproportionate impact of variety

on welfare found in the previous literature suggests that antitrust enforcers and policy makers

should weigh potential changes in variety more than potential changes in price. We find that

although the gains from access to variety are still meaningful, they are not significantly greater

than the estimated gains from lower prices due to e-commerce. For example, if online retail has

led to a 5% decrease in prices, we find that lower prices would account for just over 50% of the

consumer welfare gains derived from online retail. That is, we estimate the variety effect to be

about equal in size to the price effect, whereas the previous literature estimates the variety effect

to be 7 to 10 times larger than the price effect. A second, related implication is that such a large

increase in welfare from variety suggests consumers could endure a significant negative income

shock and still be as well off as before online retail. In other words,the compensating variation of

the additional variety is negative and large in magnitude. Thus, an implication of Brynjolfsson,

Hu, and Smith (2003) is that online retail has led to a massive decline in the price index for books.

If this effect holds generally across online retail sectors, this may suggest the Consumer Price

Index (CPI) has also seen a rapid decline. However, our results suggest this is unlikely to be the

case.

1A large body of literature that has highlighted across-market differences in demand,including Waldfogel (2003),

Waldfogel (2004), Bronnenberg,Dhar, and Dube (2009), Choi and Bell (2011), and Bronnenberg, Dube, and Gentzkow

(2012).

C

The RAND Corporation 2018.

QUAN AND WILLIAMS / 879

Toobtain these estimates, we must confront empirical challenges that arise from infrequently

purchased products. Although demand estimation techniques, such as Berry (1994) and Berry,

Levinsohn, and Pakes (1995), have been very successful in producing sensible estimates with

aggregated data (across geographic markets, time, and/or products), it is typically problematic to

apply these techniques to very granular data. With high-frequency sales data, such as our own,

the number of available options typically rises as fast (or faster) than the number of purchases.

As a result, for any given product, we may observe few, if any, sales and this will be the case for

thousands of products. This means if we take the observed market shares as proxies for the true

underlying choice probabilities, they will likely be observed with error. Because the true und-

erlying choice probabilities are unknown, this small sample issue is readily apparent to the

researcher only when no sale, opposed to few sales, is observed.

Two approaches are commonly used to force data that suffers from small sample sizes into

existing estimation techniques. However, we show both are unsatisfactory when the goal is to

estimate the demand for narrowly defined products across narrowly defined markets. The first

is to aggregate data over markets or products until observations with zero sales disappear. We

show that aggregation exactly smooths over the heterogeneity of interest. The second approach

is to ignore the issue and simply omit observations with zero sales from the analysis (such as,

focusing on just popular products). Omitting observations without sales assumes that there is no

demand for these products and is problematic for two reasons. First, it creates a selection bias

in the demand estimates (Berry, Linton, and Pakes, 2004; Gandhi, Lu, and Shi, 2013, 2014),

which tends to result in estimating consumers as too price inelastic. Second, for our setting, this is

particularly problematic because if uncorrected, we would overstate the degree of heterogeneity

across markets (Ellison and Glaeser, 1997), leading us to understate the gains from online variety.

For example, if we observe only one shoe sale in a particular market, it would suggest there are

no gains from additional variety because only one particular product is desired.

Weaddress these concerns by augmenting the nested logit model with across-market random

effects, which allowsus to decompose product-level unobservables into a mean national-product-

level fixed effect and local market deviations. With a parametric assumption on the deviations,

we show that aggregating to the national level leads to an estimating equation that involves

only national-product-level market shares, local-nest market shares, and the distribution of local

deviations. Importantly, the estimator does not rely on the local market shares of individual

products, the vast majority of which are zero at each location, but allows us to retain information

about the distribution of local demand heterogeneity. Our results lie between the extremes of

dropping all of the zeros and adjusting all of the zeros by the same amount.2As a result,

for example, the unsold boot in Alaska is treated differently than the unsold sandal in Alaska.

Additionally,the method addresses the well-known econometric challenge that logit-style demand

models tend to overstate welfare gains under largechanges in the choice set, as each new product

introduces a new dimension of unobserved consumer heterogeneity (Ackerberg and Rysman,

2005). We compare our estimates to existing approaches and show it performs well.

To identify the distribution of the random effects, we appeal to what is commonly viewed

as a data problem—zero shares—as the source of identification. Note that our random-effects

model does not by itself predict or explain zero market shares, rather, local zeros are rationalized

by modelling the finite multinomial that generates purchases at the local level. Our across-market

heterogeneity parameters are then chosen to match the model’s predictions to a set of micro

moments (Petrin, 2002; Berry, Levinsohn, and Pakes, 2004) that are based on the observed

fraction of local zeros. For example, if a product experiences a large number of sales but also

many locations with no sale, this would suggest that the demand for this product is concentrated

in select markets and the preference heterogeneity for this product is high. The design of our

2For example, a common approach called the Laplace transformation adds a single sale to all products and

recalculates shares. Gandhi, Lu, and Shi (2014) improves upon this by showing that there is an optimal market share

adjustment that minimizes the asymptotic bias of the estimator.

C

The RAND Corporation 2018.