Can switching to an open model actually cut your AI costs?
On the narrow, high-volume tasks, yes. Intercom was spending $250,000 a month running one summarization job through a frontier API and replaced it with a fine-tuned 14-billion-parameter Qwen model. The decision is made per task rather than per platform: the hardest prompt in the same product still runs on a frontier model.
Open Source AIEnterprise AIAI Infrastructure
1. Pick the task, not the stack
Intercom’s $250,000-a-month line item was one job: query canonicalization, a summarization task running at volume. That is the profile where a smaller fine-tuned model wins: narrow, repetitive, well-specified, and enormous in aggregate. The hardest prompt inside the same product, the one that answers the end user’s question, still runs on a frontier model in production. Chief AI Officer Fergal Reid moved one task off the API and left the rest where it was.
2. The open weights had to get good enough first
Reid dates the shift to around DeepSeek’s release, when open-weight models got close enough to closed ones to be viable for core tasks. That threshold moved, which means the answer to this question carries a date. The Qwen 3 models were the ones that back-tested well for Intercom’s workload, and Reid is explicit that a large training expense was absorbed by whoever released the weights.
3. The savings are partly a headcount trade
Intercom’s AI team went from 10 people to 55 in about two and a half years, with plans to double again. Their standard training setup is a node of H200s, and the approach moved from LoRA and other parameter-efficient methods toward distributed full fine-tuning and reinforcement learning. If you count only the inference bill you will get the decision wrong: you are buying an in-house post-training capability and the people who operate it.
4. Do not let a benchmark make the call
Reid’s warning is Goodhart’s law in its native habitat. It is easy for a researcher optimizing a benchmark to teach a model the benchmark’s task and then watch it fail to generalize. Intercom’s discipline is their own back tests plus A/B testing in production, run by people hired as scientists rather than only as engineers. A public leaderboard is not evidence about your workload.
Why it matters
“Open source is cheaper” is not a strategy, and neither is “frontier models are better.” The teams saving real money are profiling their own traffic, finding the one or two tasks that are high-volume and narrow, and moving those. Everything else stays where it works.
From the conversation
This explainer is drawn from these episodes — each carries its full transcript.
