Unknown until Known
| DOI | http://doi.org/10.1002/jcaf.22333 |
| Date | 01 April 2018 |
| Author | Tim Chartier |
| Published date | 01 April 2018 |
139
© 2018 Wiley Periodicals, Inc.
Published online in Wiley Online Library (wileyonlinelibrary.com).
DOI 10.1002/jcaf.22333
C
o
m
m
e
n
t
a
r
y
Unknown until Known
Tim Chartier
Suppose we are sitting on
a couch flipping through
television channels and
stop to watch a clip of the
1992 film “My Cousin Vinny.”
I comment, “I love this film!”
Would you have other films
that you could recommend that
you expect I would like, too?
Such a knack for recommend-
ing could have earned you a
million dollars from Netflix
about a decade ago.
To enter the millionaires’
club, you needed to do much
more than recommend a film
to me on the couch. In a sense,
you had thousands and thou-
sands of people sitting on the
couch telling you the films they
liked and how much they liked
them. Rather than a couch,
you had access to Netflix rat-
ings data. To win the case, you
needed to do a “better” job of
predicting than Netflix’s rec-
ommendation system, called
CinematchSM, which predicted
whether someone would enjoy
a movie based on how much
they liked or disliked other
movies. Specifically, if your
predictions are at least 10%
better than CinematchSM
for this training set, you won
the Netflix Prize worth a mil-
lion dollars! Oh, there was
a caveat—you had to share
the results with Netflix and
“describe to the world how you
did it and why it works.”
The competition began
on October 2, 2006. Netflix
provided a training data set
of 100,480,507 ratings that
480,189 users gave to 17,770
movies. Each training rating
was a quadruplet of the form
grade>. The user and movie
fields were integer IDs, while
grades were from one to five
(integral) stars.
By October 8, a team called
WXYZConsulting had already
beaten Cinematch’s results.
By October 15, there were
three teams who had beaten
Cinematch. Within 2 weeks,
the (at that time) DVD rental
company had received 169 sub-
missions. By June 2007, over
20,000 teams had registered
for the competition from over
150 countries.
The quick improvement
in those opening weeks made
Netflix’s task look simple.
But, fortunes changed as the
rate of improvement slowed.
Month after month, the same
three or four teams topped
the leaderboard, ticking off
decimal by decimal improve-
ment. After a year, BellKor, a
research group from AT&T,
was in first place besting Cin-
ematch by 8.43%. At the same
time, murmurs spread about
the potential impossibility of
10% improvement.
Suddenly, in November
2007, a new entrant, “Just a
guy in a garage,” appeared with
a score that not only popped
into the top 10 but performed
at a level that had taken
BellKor 7 months to achieve.
By mid-January, there were
just five teams, out of 25,000
entrants, ahead of the garage
guy entrant. Adding to the
mystery, no one knew the iden-
tity of the entrant or what sta-
tistical sorcery were sprinkled
into his methods.
Turns out the entrant,
Gavin Potter, didn’t work out
of a garage but instead a back
bedroom of his London home.
The 48-year-old Englishman
was a retired management con-
sultant with an undergraduate
degree in psychology and a
master’s in operations research.
As he developed his predictive
algorithm, Potter turned to his
oldest daughter, who would
break from her high school
homework to serve as her
father’s math consultant and
solve various tasks requiring
Calculus. Potter’s improvement
came not through advanced
To continue reading
Request your trial