Fergal Reid
- 1 episode on Chain of Thought
- 5 topics covered
- 1 full transcript
- Feb 2026 first on the show
Fergal Reid is the Chief AI Officer at Fin (Intercom), where he scaled the AI team from 10 people to 55. On Chain of Thought he walked through how Intercom cut a $250K-per-month summarization bill by replacing GPT with a fine-tuned Qwen model, and how the surrounding systems, not the core LLM, drove most of the gains in their Fin agent.
Episodes
What the conversations cover
These notes summarize the episodes featuring Fergal Reid, including other speakers in panel conversations. Follow a source link to read the discussion in context.
EP 49 How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid
- Intercom was spending an estimated quarter-million dollars a month — Fergal Reid’s own hedge: “I don’t remember exactly, but it could have been like a quarter million dollars a month” — on a single query-canonicalization task running on GPT-4.1. A fine-tuned 14B-parameter Qwen 3 model replaced it, saving “almost all of those, you know, several hundred thousand a month on inference just by doing that one thing.” Episode takeaways
- Fin’s resolution rate climbed from 30% at launch to nearly 70% — and the vast majority of that gain came not from the core frontier LLM but from surrounding systems: a custom retrieval model fine-tuned on production resolution signals, a custom re-ranker built on ModernBERT that outperformed Cohere’s best re-ranker, and query canonicalization upstream of retrieval. Episode takeaways
- Intercom’s “hard resolution” metric — where an end user explicitly confirms their question was answered — is the North Star that prevents building a mere deflection machine. Fergal described the ratio of soft to hard resolutions as something that “should be roughly constant for every model change,” and a multi-million-interaction A/B test validated their switch from GPT to Claude Sonnet. Episode takeaways
In their words
Not directly because it requires a human in the loop and when we're doing RLHF it's offline rather than in production. We don't have an open loop RL in production.
Fergal Reid How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid · 20:56 Transcript excerpts; ellipses mark omitted words. Some automatically selected quotes have filler words removed.
Explained on AI, decoded
- Is an open-weight AI model the same as an open-source one?
- What is a reranker in a RAG system?
- Can switching to an open model actually cut your AI costs?
- When should you use a small language model instead of a frontier model in production?
- Should you use prompting, RAG, or fine-tuning to customize an AI model?
- Are small language models better than large ones for production?
- How do you cut the cost of running an AI agent?
- What's the difference between AI observability, evaluation, and benchmarking?
- What is a token in AI, and why does it decide what AI costs?