The Bitter Lesson

Sutton’s examples are concrete. Chess programs built around grandmaster knowledge lost to deep search on fast hardware. Go programs full of human strategy lost to systems that learned from self-play. In speech recognition and computer vision, hand-designed features gave way to statistical and then deep learning methods trained on more data. His point is that the field’s steady gain in cheap computation, the trend described by Moore’s law, rewards approaches that scale with it.

A business version: a team spends months writing rules to route support tickets, then a general model fine-tuned on past tickets does better and needs less upkeep when the product changes. The rules encoded what people knew; the learned system kept improving as data and compute grew.

The essay is an argument, not a proven law, and it is often stretched. Sutton was writing about research methods over decades, not saying that domain knowledge is worthless in a product. Teams still need good data, evaluation and judgment about what to build. Our editorial reading: the lesson warns against betting a system on clever scaffolding that a larger general model may soon make unnecessary. It sits close to scaling laws, which measure how model quality improves with more compute, data and parameters.

Guests apply it to their own work. In episode 12, at 13:03, Logan Kilpatrick, then at Google, describes the bitter lesson as the story of a general-purpose approach beating a specialized one, and connects it to lowering the barrier for developers. The episode was sponsored by Galileo, which Conor mentions at 18:07. In episode 63, at 18:42, Dhruv Batra cites his team’s post “The Bitter Lesson of Web Agents” while arguing that agents which read screens like people will scale further than custom scripts for each website. Both are the guests’ views on where the lesson leads. For more on Batra’s argument, see Why Won’t the Web Get APIs for AI Agents?.