Netflix Prize 2009: BellKor wins the $1 million recommender contest

Netflix awarded $1 million on 21 September 2009 to BellKor's Pragmatic Chaos after a three-year contest to beat the Cinematch recommender by 10% on RMSE. Matrix factorization and RBMs shaped both research and production.

Event date: Published:
HistoryResearch

On 21 September 2009, Netflix held an award ceremony for the Netflix Prize, a $1 million open competition that had started on 2 October 2006. The company released about 100,480,507 ratings from roughly 480,189 users on 17,770 movies and asked teams to beat its Cinematch baseline by 10% on RMSE. The winning team BellKor’s Pragmatic Chaos reached test RMSE 0.8567; The Ensemble tied the score but submitted about 20 minutes later under contest rules.

Factorization, ensembles, and neural nets

Top solutions blended matrix factorization, ensemble methods, and clever feature engineering. Yehuda Koren, Robert M. Bell, and Chris Volinsky documented industrial collaborative filtering for Netflix in IEEE Computer (2009). Earlier, Ruslan Salakhutdinov, Andriy Mnih, and Geoffrey Hinton showed at ICML 2007 how restricted Boltzmann machines (RBMs) could model user ratings; their Machine Learning at Toronto team won the 2007 Progress Prize portion of the contest.

Netflix later wrote that it shipped ideas from the Progress Prize work, including singular value decomposition and RBM components, into production, while the full grand-prize ensemble was too costly to deploy as-is. The contest still popularized recommender systems as a flagship machine learning application alongside methods such as random forests and support vector machines.

Privacy backlash and an cancelled sequel

Researchers Arvind Narayanan and Vitaly Shmatikov demonstrated in 2007 that anonymous Netflix ratings could be re-identified by cross-matching public IMDb profiles. A class action filed in December 2009 over privacy concerns settled in March 2010, and Netflix cancelled a planned second contest.

Why it matters

The Netflix Prize turned supervised learning on sparse user-item data into a spectator sport and connected classical factorization to Boltzmann machine ideas from the deep belief network era. It sits between ensemble classics like Breiman’s random forests and the GPU-accelerated deep learning wave that followed.