Skip to content

Fergal Reid

Intercom

  • 1 episode on Chain of Thought
  • 5 topics covered
  • 1 full transcript
  • Feb 2026 first on the show
Fergal Reid

Fergal Reid is the Chief AI Officer at Fin (Intercom), where he scaled the AI team from 10 people to 55. On Chain of Thought he walked through how Intercom cut a $250K-per-month summarization bill by replacing GPT with a fine-tuned Qwen model, and how the surrounding systems, not the core LLM, drove most of the gains in their Fin agent.

Cover art for How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid

Episodes

What the conversations cover

These notes summarize the episodes featuring Fergal Reid, including other speakers in panel conversations. Follow a source link to read the discussion in context.

EP 49 How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid

  • Intercom was spending an estimated quarter-million dollars a month — Fergal Reid’s own hedge: “I don’t remember exactly, but it could have been like a quarter million dollars a month” — on a single query-canonicalization task running on GPT-4.1. A fine-tuned 14B-parameter Qwen 3 model replaced it, saving “almost all of those, you know, several hundred thousand a month on inference just by doing that one thing.” Episode takeaways
  • Fin’s resolution rate climbed from 30% at launch to nearly 70% — and the vast majority of that gain came not from the core frontier LLM but from surrounding systems: a custom retrieval model fine-tuned on production resolution signals, a custom re-ranker built on ModernBERT that outperformed Cohere’s best re-ranker, and query canonicalization upstream of retrieval. Episode takeaways
  • Intercom’s “hard resolution” metric — where an end user explicitly confirms their question was answered — is the North Star that prevents building a mere deflection machine. Fergal described the ratio of soft to hard resolutions as something that “should be roughly constant for every model change,” and a multi-million-interaction A/B test validated their switch from GPT to Claude Sonnet. Episode takeaways

In their words

Not directly because it requires a human in the loop and when we're doing RLHF it's offline rather than in production. We don't have an open loop RL in production.

Fergal Reid How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid · 20:56

Transcript excerpts; ellipses mark omitted words. Some automatically selected quotes have filler words removed.