The Bitter Lesson

#The essay in one paragraph

In March 2019, Rich Sutton, one of the founders of modern reinforcement learning, published a short essay called The Bitter Lesson. His claim, drawn from about 70 years of AI history, is that general methods that can make use of more and more computation end up beating methods built on human knowledge, and usually by a wide margin. The reason is economic, not philosophical: the cost of computation keeps falling, so a method that gets better with more compute keeps improving year after year, while a method that encodes what experts know improves only as fast as the experts do. He names two such general methods: search and learning.

#Why it’s “bitter”

The pattern Sutton describes repeats in a recognizable cycle:

1. Researchers build their own understanding of the problem into the system.
2. It works, it’s satisfying, and it wins in the short term.
3. It plateaus, and sometimes it even blocks progress.
4. A cruder-looking method that leans on scale overtakes it.

The bitterness is that the winning approach succeeds without the insight people worked hard to encode. It can feel like the field’s cleverness was wasted.

#His four examples

• Chess. The program that beat Kasparov in 1997 relied on massive, deep search rather than encoded chess knowledge. Researchers who had built knowledge-based systems were largely unconvinced by how it won.
• Go. About twenty years later, the same story played out with search plus learning from self-play, after decades of effort to exploit human understanding of the game.
• Speech recognition. In a 1970s DARPA-sponsored competition, systems built on human knowledge of words, phonemes, and the vocal tract lost to statistical methods such as hidden Markov models, which used more computation. Deep learning later continued the same trend.
• Computer vision. Hand-designed ideas such as edge detection, generalized cylinders, and SIFT features gave way to deep convolutional networks, which build in only a little structure (convolution and certain invariances) and learn the rest.

#The second, deeper lesson

Sutton goes further than “compute wins.” He argues that the actual contents of minds are enormously and irreducibly complex, so we should stop trying to hand-write simple descriptions of them, such as how to think about space, objects, or symmetries. What we should build in are the meta-methods that can discover that complexity on their own. The goal is agents that can discover things the way we do, not agents that contain what we’ve already discovered.

#What the lesson does not say

The essay is often quoted more loosely than it was written. Three common misreadings:

• “No structure is ever allowed.” Sutton himself credits convolution and invariances. The lesson argues against encoding solutions, not against all design.
• “Bigger always wins in the short run.” It’s a long-run claim. Hand-built methods often win for years first, which is exactly why the lesson keeps having to be relearned.
• “Everyone agrees.” It’s a contested view. Rodney Brooks replied shortly after with a counter-essay arguing that human-designed structure, such as convolutional architectures themselves, is part of why learning works at all.

#A checklist for papers that invoke it

When a paper cites the Bitter Lesson, these are the questions worth asking:

1. Does the method improve with scale? The lesson is about methods that get better as models or compute grow. Is there evidence of that trend, not just one data point?
2. Which human priors were removed, and which remain? Removing one hand-designed component often moves the human design somewhere else, such as into prompts, signals, or recovery rules.
3. Is the comparison fair on compute? “General method wins” means more if it wins at equal or lower cost, not only when given more of it.
4. Is it search, learning, or both? Name which mechanism is doing the work.

#Where this shows up in the CLM paper

The paper’s claim maps onto the essay directly. The human prior is the hand-designed context policy: compaction thresholds, prescribed summarization prompts, and fixed tool menus (see the refresher on compaction, offloading, and retrieval). The general method is giving the model an editable context and letting it find the policy, through search (in-context exploration and skill evolution) and learning (reinforcement learning). The checklist above is how §1 of this review evaluates that claim.