Naive Bayes spam filter: count words, roll the dice

Feed labeled spam and ham lines, tweak Laplace smoothing, and watch a 1990s-style bag-of-words classifier score your message, with per-word log-likelihood tilts on screen.

y26w43

Last week a Hopfield net remembered faces with physics. This week, no neurons at all; just word counts and a bold lie called conditional independence. That lie powered inbox filters for years before giant deep-learning stacks took over vision and language.

Naive Bayes treats each email as a bag of tokens. You show it spam and ham examples; it estimates how often each word appears in each class, multiplies smoothed likelihoods (Laplace saves you from zero counts), and picks the bigger posterior. It is classic supervised learning: labeled lines in, class label out, the same statistical spirit as much of machine learning and everyday NLP, only transparent enough to read off the screen.

How to play:

  1. Skim the spam and ham training lines (edit them or add your own, one message per line).
  2. The message box starts with a spam example from the training set; press Classify (or Try spam example / Try ham example).
  3. Drag Smoothing and watch rare words stop dominating (or start mattering).
  4. Read Word contributions: which tokens tilt toward spam vs ham?

Try this:

  1. Hit Classify on the default line (“win free money click here now”): you should see a strong spam verdict.
  2. Click Try ham example (“team meeting moved to tuesday afternoon”): ham should dominate.
  3. Type a mixed sentence and watch the bars wobble. Empty message? The gadget nags you politely.

Why this matters

Paul Graham’s 2002 essay helped popularize Bayesian spam filtering for indie developers; filters like SpamAssassin mixed rules with statistics. The “naive” assumption is wrong (words are not independent), yet the trick often works because the decision boundary only needs to be good enough. Modern LLMs are galaxies away, but the intuition survives: features you can count plus probabilities you can explain, right down to individual tokens.

Missed the earlier editions? Train a perceptron, watch a Markov chain babble, lose to unbeatable tic-tac-toe, teach a filter to see edges, watch an agent learn from reward, see how a model reads text in tokens, watch one roll downhill to learn, let a machine find groups on its own, see a line learn to bend, watch words decide who to listen to, explore vs exploit on slot machines, chat with ELIZA, or scribble Hopfield memory.

Each line is one labeled email. The model counts words per class, assumes words are independent (the “naive” part), and multiplies smoothed likelihoods. Drag smoothing to see rare words matter less or more.