Q-learning: watch an agent learn from reward
Drop an agent in a grid with a goal and a trap. It starts knowing nothing and learns by trial and error which way to go, one reward at a time. The idea behind AlphaGo and RLHF.
This week the machine learns the hard way: by trial, error, and reward. The perceptron adjusted a line, convolution filtered pixels, tic-tac-toe searched ahead. Today’s agent gets none of that. It is dropped into a world it does not understand and told only one thing: some outcomes feel good, others bad.
The gadget below is a tiny gridworld. There is a start, a goal worth +1, and a trap worth -1. The agent can move up, down, left, or right. It has no map and no rules, only a memory of how rewarding each move from each square has turned out. That memory is a table of numbers called a Q-table, and the method for filling it in is Q-learning.
Try this:
- Press Train. The agent plays hundreds of quick episodes, mostly stumbling around at random. Watch the squares light up: value spreads backward from the goal, square by square, like a scent trail.
- The arrows show the agent’s current best guess for which way to go from each square. As training continues they line up into a route that reaches the goal and steers clear of the trap.
- Press Run agent to watch it follow what it learned. Then Reset and train again; the random exploration means the path can come out different each time.
Reward is the only teacher
This is reinforcement learning: no labelled examples, no human showing the right answer, just a reward signal and a lot of practice. The update rule is deceptively simple. After each move the agent nudges its estimate for that square-and-direction toward “the reward I just got, plus how good the next square looks”. Repeat that thousands of times and a competent policy emerges from pure feedback.
From gridworld to Go and chatbots
Scale this idea up and it becomes some of the most famous AI of the decade. AlphaGo reached superhuman Go by combining search with reinforcement learning and self-play, the story told in AlphaGo vs Lee Sedol (2016). The same family of methods, as RLHF, is how chatbots are tuned to be helpful, and how reasoning models like DeepSeek-R1 are trained to think before they answer. Different worlds, same core loop: act, get a reward, update.
Your gridworld agent is the smallest honest version of that loop. It knows nothing, and reward alone teaches it the way.
Missed the earlier editions? Train a perceptron, watch a Markov chain babble, lose to unbeatable tic-tac-toe, or teach a filter to see edges.