AlphaGo Zero: superhuman Go with no human games (2017)

Published online on 19 October 2017, DeepMind''s AlphaGo Zero learned Go entirely from self-play, beating the version that defeated Lee Sedol without using any human master games as training data.

Event date: Published:
HistoryResearch

On 19 October 2017, Nature published online Mastering the game of Go without human knowledge from DeepMind. AlphaGo Zero learned Go from scratch: no human professional games, only self-play and reinforcement learning. In three days it surpassed the AlphaGo that had beaten Lee Sedol in 2016 (article); in 40 days it reached levels that earlier systems had needed months of human data to approach.

Tabula rasa self-play

The earlier AlphaGo pipeline started from millions of human moves, then refined with self-play. AlphaGo Zero removed that first stage. A single deep neural network combined policy and value heads and guided Monte Carlo tree search against itself. Stronger play generated stronger training targets, a feedback loop Demis Hassabis’s team had pioneered in games research, now pushed to an extreme where human teaching was optional.

David Silver, Julian Schrittwieser, and a large DeepMind co-author list reported that Zero beat the Lee Sedol version 100 games to 0, using a simpler stack and less compute than the 2016 match system. The same recipe later produced AlphaZero, which generalized self-play to chess and shogi from random play.

Same year as the Transformer

2017 was crowded with milestones. While Zero was mastering Go without human knowledge, Google researchers published the Transformer for NLP, a different domain but the same lesson: scale, data (even if self-generated), and deep learning could leap past hand-engineered pipelines.

Why it matters

AlphaGo Zero showed that expert human datasets are not always the best teacher. Self-play at scale can discover strategies humans never recorded, which matters beyond board games whenever simulation is cheap and reward is clear. It closed the loop opened by the Lee Sedol match and set expectations for later reinforcement learning systems in robotics, planning, and alignment research.

#reinforcement-learning#deep-learning