Neural network: watch a straight line learn to bend
One neuron can only draw a straight line, so it gets stuck on XOR and the circle. Add a hidden layer, press Train, and watch gradient descent bend the boundary into curves. This is why deep learning needs depth.
Way back in the first edition, a single perceptron learned to split two groups with a straight line. Then we watched gradient descent roll downhill to tune weights. This week we put the two together and hit the wall that once nearly killed AI: some things a straight line simply cannot separate.
The gadget below is a tiny neural network. Two colors of dots, one hidden rule that tells them apart. You choose the data and the model, then press Train. The colored background is the network’s current decision, and it reshapes live as gradient descent nudges the weights.
Try this:
- Leave it on Circle with Hidden layer and press Train. Watch the boundary curl into a loop that wraps the inner dots. A network with a hidden layer can bend.
- Now switch the model to One neuron and Train again. One neuron is a perceptron: it can only draw a single straight line, so it gets stuck around 50 to 75 percent and never closes the circle.
- Switch the data to XOR (opposite corners share a color). One neuron fails here too, the textbook example. Flip back to Hidden layer and watch it carve the plane into the right four regions.
Why this matters
In 1969 a famous book showed that a single perceptron cannot learn XOR. That limitation, and the hype it punctured, helped tip the field into the first AI winter. The fix was not a smarter single neuron but depth: stack a hidden layer of neurons and the network can combine simple straight cuts into curved, nonlinear boundaries. That is the whole idea of a neural network, and of deep learning once you stack many layers. That depth became trainable in 1986 with backpropagation.
The bending you just watched is the same trick, scaled up with fast hardware and huge datasets, that let AlexNet recognize photographs decades later. Behind the magic is a straight line that learned to bend.
Missed the earlier editions? Train a perceptron, watch a Markov chain babble, lose to unbeatable tic-tac-toe, teach a filter to see edges, watch an agent learn from reward, see how a model reads text in tokens, watch one roll downhill to learn, or let a machine find groups on its own.