Deep belief network
Deep neural network built by greedily stacking restricted Boltzmann machines, then fine-tuned with backpropagation; Hinton et al. 2006 helped restart the deep-learning revival.
A deep belief network (DBN) is a deep neural network trained by greedily stacking restricted Boltzmann machines (RBMs) layer by layer, then fine-tuning the full stack with backpropagation. Geoffrey Hinton, Simon Osindero, and Yee-Whye Teh published the fast learning algorithm in Neural Computation in July 2006; Hinton and Ruslan Salakhutdinov demonstrated high-quality dimensionality reduction in Science the same month.
DBNs showed that deep nets could be trained again after years of scepticism, helping launch the deep learning revival later accelerated by ImageNet and AlexNet. Later systems often skipped pretraining in favour of ReLU, GPUs, and end-to-end supervised learning. See the article and timeline entry.