DQN on Atari: DeepMind learns to play from pixels (2013)
On 19 December 2013 DeepMind researchers posted "Playing Atari with Deep Reinforcement Learning" on arXiv, combining a convolutional network with Q-learning, experience replay, and target networks to control arcade games from raw screen input.
On 19 December 2013, researchers at DeepMind led by David Silver and Volodymyr Mnih posted Playing Atari with Deep Reinforcement Learning on arXiv (1312.5602). The system, later known as a deep Q-network (DQN), learned to play classic Atari 2600 games from pixels alone, using the same controller interface a human would, without hand-crafted features for each title.
One network, many games
Earlier reinforcement learning (RL) often relied on carefully engineered state descriptions. DQN instead fed stacked grayscale frames into a convolutional neural network (CNN) that approximated action values for Q-learning. A single architecture could be trained on dozens of games, with only the final output layer differing per game. The arXiv report showed human-level scores on several titles and clear learning curves on others.
Two stabilisers made neural network Q-learning practical. Experience replay stored transitions in a buffer and sampled mini-batches at random, breaking correlations between consecutive frames. A separate target network, updated slowly, stopped the moving-target problem that had plagued earlier deep RL attempts. See the experience replay glossary entry for how replay fits modern RL pipelines.
From arcade cabinets to Go
The Atari result was an early public signal that DeepMind could pair deep learning with RL at scale, the same recipe that later powered AlphaGo against Lee Sedol. While Montreal researchers were opening generative modeling with GANs in 2014, London was showing that a CNN could learn control policies directly from high-dimensional sensory input.
A Nature paper in 2015 consolidated the method and reported human-level performance on a large suite of games, cementing DQN as a landmark in deep RL.
Why it matters
DQN helped establish end-to-end learning from pixels as a credible alternative to hand-built game AI. Experience replay and target networks became standard ingredients in later algorithms. The line from Atari scores to championship Go and beyond runs through the same combination of representation learning, trial-and-error signal, and compute that defines much of modern reinforcement learning research.