Experience replay
Reinforcement-learning technique that stores past transitions in a buffer and trains on random mini-batches, decorrelating updates and reusing rare events.
Experience replay keeps a replay buffer of past environment transitions (state, action, reward, next state) and samples mini-batches from it during training. Random sampling breaks temporal correlation between consecutive frames or steps, which otherwise destabilises neural network updates in reinforcement learning. Revisiting stored data also improves sample efficiency when new experience is costly.
Long-Ji Lin described early self-improving agents with stored experience in 1992. DeepMind’s DQN for Atari (article) popularised replay together with a target network for deep Q-learning. Later algorithms such as prioritized replay and off-policy actor-critic methods extend the same idea. See the timeline entry.