Frontier models are built to be good at nearly everything, and every answer from one costs the computing power of a very large model. A small language model trades breadth for cost and speed. With fewer parameters, it is usually cheaper and faster per answer, the smallest can run on a laptop or phone, many fit on a company’s own servers, and it is often post-trained to do one job well.
The usual pattern is to prototype on a frontier model, measure quality with evals, then move stable, high-volume tasks to a small model that still passes them. Intercom did that with one summarization step on Chain of Thought’s episode 49. Small language model vs. frontier model covers when each wins.
Hear it from the guest
“We're going to have a compute crunch. And I think out of necessity, that's going to force us to start to work, really work on small language models.”
Sterling Chin, Postman
EP 50 · around 56:02 · read the transcript
Quotes lightly edited to remove filler words.
Go deeper
- When should you use a small language model instead of a frontier model in production? AI, decoded · Small Language Model vs. Frontier Model
- Are small language models better than large ones for production? AI, decoded · Small Language Models in Production
- Can switching to an open model actually cut your AI costs? AI, decoded · When an Open Model Beats a Frontier API on Cost
From the conversation
-
How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid - I Built an AI Coworker That Runs 90% of My Day | Sterling Chin, Postman
- The AI Agent Trust Gap: Bridging Risk to Reliability | Elastic’s Philipp Krenn
- Explaining Eval Engineering | Galileo's Vikram Chatterji
- Architecting Reliable Agentic AI | Cisco’s Giovanna Carofiglio on the AGNTCY Collective
- From Demo to Defensibility: How to Build an AI Business that Lasts | Aurimas Griciūnas
- The Emerging AI Agent Stack | CrewAI’s João Moura
- Vercel's Playbook for AI Agents: From Vibe Check to Production | Malte Ubl