LeCun's zip code nets: when convolution met backpropagation (1989)

In 1989, Yann LeCun, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne Hubbard, and Lawrence D. Jackel published Backpropagation Applied to Handwritten Zip Code Recognition in Neural Computation (volume 1, issue 4). Working at AT&T Bell Laboratories, they trained a convolutional neural network (CNN) with backpropagation on digits from the US Postal Service zip code database. The system learned local feature detectors that could recognise handwritten numerals at useful accuracy, at a time when much of the field was still recovering from the second AI winter (article).

Local filters instead of full connections

Earlier multilayer neural networks connected every input pixel to every unit in the first hidden layer, which is expensive for images. The Bell Labs design reused the same small convolutional kernels across the image, sharing weights and respecting spatial structure. Gradient descent through backpropagation adjusted those kernels end to end. The architecture foreshadowed LeNet, the family of convolutional nets LeCun would refine through the 1990s.

From lab demo to cheque readers

The 1989 result did not end the winter overnight, but it showed that supervised learning on real postal data could work in practice. By the mid-1990s, similar convolutional systems were deployed to read digits on bank cheques in the United States, one of the quiet commercial successes cited in the second AI winter article. The same line of work later fed into systems of the ImageNet era such as AlexNet (article).

Why it matters

The paper linked two ideas that still define modern vision: convolution for efficient spatial features and backpropagation for learning them from data. LeCun shared the 2018 Turing Award with Geoffrey Hinton and Yoshua Bengio for work including this convolutional line. Readers can explore the operation interactively in Geek of the Week: Convolution playground.