Chain of Thought — Episodes

The First Fully Autonomous AI Attack Is 18 Months Away | Kristin Lovejoy

Thu, 11 Jun 2026 11:30:00 GMT

Kris Lovejoy, Global Head of Strategy at Kyndryl, the world's largest IT infrastructure services provider, predicts the first fully autonomous AI attack will land within 18 months. She breaks down why 62% of enterprise AI initiatives are stuck in pilots and what a real control plane for agentic AI requires.

The AI Framework Era Is Over: Why Context Is the Moat | Jerry Liu

Wed, 03 Jun 2026 11:30:00 GMT

Jerry Liu built LlamaIndex into one of the most installed AI frameworks of the last three years, then bet the company that the framework era is over. He explains why context quality is the moat that survives as agent loops get good enough.

We Built Agents, Nobody Built HR | Tyler Akidau, Redpanda

Wed, 27 May 2026 11:00:00 GMT

Tyler Akidau, CTO of Redpanda and author of the O'Reilly Streaming Systems book, makes the case that enterprises are shipping AI agents into production without the governance layer they need. He lays out the four pillars of agent HR (identity, authorization, observability, accountability) and why inline enforcement via CLAUDE.md fails the moment the stakes get real.

How Superhuman Built AI Into a 100ms Product | Loïc Houssier

Fri, 22 May 2026 11:00:00 GMT

Loïc Houssier, VP of Engineering at Superhuman (the email client Grammarly acquired for $825M in July 2025), explains how his team retrofitted AI features into a product whose entire brand promise is sub-100ms speed. He breaks down the model-routing strategy, the eval framework his PMs own, and why his team auto-drafts every reply but refuses to auto-send any of them.

The AI Hiring Doom Loop: Applications Up 239%, Hires Down 75%

Wed, 06 May 2026 11:30:00 GMT

Daniel Chait, CEO of Greenhouse (the hiring platform behind 22 million applications a month), coined the term "AI doom loop": applications up 239% since ChatGPT, but 75% fewer reach the hire stage. Inside: why software engineers are the worst auto-appliers and how Greenhouse is rebuilding hiring.

Every AI Agent Has an Evaluation Gap | Alex Ratner, Snorkel AI

Wed, 29 Apr 2026 11:58:48 GMT

Snorkel CEO Alex Ratner maps the evaluation gap blocking AI agents from real enterprise work, walks through the company's $3M Open Benchmarks Grant, and explains why pure 'environment' vendors don't actually understand how AI works.

250,000 Lines of Code/Week: Inside an AMD VP's Agent-First Workflow | Anush Elangovan

Wed, 22 Apr 2026 11:00:00 GMT

AMD's VP of AI Software runs 10-12 Claude Code agents in parallel, burns 6.5 billion tokens a week, and rewrote a 25-year-old Slurm replacement in Rust overnight. Anush Elangovan on why normal SDLC is dead, testing is the new code review, and software is just tokens.

Hallucinations Are a Data Architecture Problem | Sudhir Hasbe, Neo4j

Thu, 16 Apr 2026 11:00:00 GMT

Sudhir Hasbe is President and Chief Product Officer at Neo4j, the graph database company powering 84 of the Fortune 100 (Walmart, Uber, Airbus) at $200M+ ARR and a $2B+ valuation. Before Neo4j, he ran product for all of Google Cloud's data analytics services: BigQuery, Looker, Dataflow, and led the Looker acquisition.

His thesis: the hallucinations we blame on AI models are really a data architecture problem. LLMs weren't trained on your enterprise knowledge, so handing them a data lake with 10,000 disconnected tables and asking them to reason is the wrong design. The fix is knowledge graphs: feeding the model a structured map of relationships, entities, and context so it can reason over meaning, not just vector similarity.

Sudhir breaks down the five capabilities knowledge graphs unlock for enterprise AI: GraphRAG (moving accuracy from 60% to 97%), semantic mapping across siloed systems, context graphs, agent memory, and multi-hop reasoning. He explains three architecture patterns customers are actually shipping, why giving an LLM hundreds of tools makes it worse, and what Uber, EA Sports, Klarna, and Novo Nordisk are doing differently.

This is the case for treating knowledge as infrastructure.

We cover:

Why enterprise AI needs a different playbook than consumer AI
The five data asset types every agentic system needs: system of record, historical, memory, context, and reference
How GraphRAG combines vector search and graph traversal to move from 60% accuracy to 95%+
Three architecture patterns: semantic layer only, semantic map plus domain data, full consolidation (the Klarna/Kiki model)
What context graphs capture that Salesforce doesn't: the Slack and email negotiation behind every deal
Why giving an LLM hundreds of tools drops accuracy, and how Uber uses knowledge graphs as a business validation layer
What Neo4j's Aura Agent, MCP server, and A2A support mean for developers starting today

Chapters:

(0:00) Why building a self-driving car is hard
(0:22) Intro
(2:03) Hallucinations as a data architecture problem
(4:31) From models-as-core to systems-of-knowledge
(6:13) Why data lakes fail AI agents
(9:15) The five data asset types enterprise agents need
(11:46) Where basic RAG breaks down: the Spotify metadata lesson
(16:00) GraphRAG: 3x accuracy, easier development, explainability
(18:47) Semantic mapping across the enterprise estate
(19:23) Three knowledge-graph architecture patterns
(22:42) Context graphs: capturing the "why" behind decisions
(25:33) Individual vs. organizational agent memory
(28:40) Multi-hop reasoning for fraud rings and AML
(31:52) Why there are no shortcuts in enterprise AI
(36:38) What happens when you give an LLM 100 tools
(39:19) The Uber example: knowledge graph as business validation
(44:42) First mile of a 26-mile marathon
(48:32) Aura Agent, MCP server, and the A2A protocol
(50:43) Where developers should start

Connect with Sudhir Hasbe:

LinkedIn: https://www.linkedin.com/in/shasbe/
Neo4j: https://neo4j.com/
Neo4j Aura: https://neo4j.com/product/auradb/

Connect with Conor:

Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon

More episodes: https://chainofthought.show

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at:
galileo.ai/mastering-multi-agent-systems

Why LLMs Are Plausibility Engines, Not Truth Engines | Dan Klein

Wed, 08 Apr 2026 11:00:00 GMT

Dan Klein, co-founder & CTO of Scaled Cognition and ACM Grace Murray Hopper Award winner, breaks down why LLMs are fundamentally plausibility engines and how his team built APT1 for under 11 million dollars. He explains why multi-model checking fails, why benchmarks measure the wrong thing, and what it takes to ship AI that enterprises can actually trust.

Agent Memory: The Last Battleground in the AI Stack | Richmond Alake, Oracle

Thu, 02 Apr 2026 11:00:00 GMT

Richmond Alake is Director of AI Developer Experience at Oracle and one of the most concrete voices on agent memory right now. His AI Engineer World's Fair talk on architecting agent memory crossed 100,000 views, he built the open-source MemoRIS library, and he co-created a course with Andrew Ng.

In this conversation, Richmond walks through memory engineering as a distinct discipline from prompt engineering and context engineering, demos a memory-aware financial services agent that runs vector, graph, spatial, and relational search in a single query, and explains the principle that separates production-grade memory systems from prototypes: don't delete, forget. If you're building agents that need to remember anything across sessions, this is the episode.

We cover:
- Why memory engineering deserves its own name, separate from prompt and context engineering
- The two failure modes Richmond sees most: wrong mental model and deleting instead of forgetting
- Four human memory types mapped to agent architecture: working, episodic, semantic, and procedural
- Demo: AFSA, a memory-aware financial services agent with converged search across data types
- How the Generative Agents paper's decay formula (relevance + recency + importance) enables controlled forgetting
- Where context engineering ends and memory engineering begins
- Why files work for prototypes but databases win in production

Chapters:
(0:00) Memory is the last battleground in AI
(0:28) Meet Richmond Alake, Oracle's AI DevEx lead
(2:23) Why memory engineering is its own discipline
(7:57) The failure modes nobody talks about
(12:49) Demo: a memory-aware financial services agent
(18:30) Segmenting context windows by memory type
(19:22) Four human memory types mapped to agent architecture
(23:51) Procedural memory in production systems
(27:11) Don't delete, forget: implementing controlled decay (33:32) Sponsor: Galileo
(35:46) Where context engineering ends and memory engineering begins
(38:50) Is agent memory fundamentally a database problem?
(44:13) Files vs. databases: what production actually needs
(51:09) Picking your lane in the AI noise
(55:44) Richmond's courses with Andrew Ng, O'Reilly classes, and where to follow

Connect with Richmond Alake: LinkedIn: https://www.linkedin.com/in/richmondalake/
Check out his Youtube: https://www.youtube.com/@richmond_a
O'Reilly courses: https://www.oreilly.com/live-events/ai-memory-management-in-agentic-systems/0642572179274/
Diagrams from the episode: https://imgur.com/a/mMtcAtk

Connect with Conor:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon

More episodes: https://chainofthought.show

Thanks to Galileo — download their free 165-page guide to mastering multi-agent systems at http://www.galileo.ai/mastering-multi-agent-systems

Context Poisoning Is Killing Your AI Agents: How to Stop It

Wed, 25 Mar 2026 11:00:00 GMT

Michel Tricot co-founded Airbyte, the open source data integration platform with 600+ free connectors that hit a $1.5 billion valuation. Now he's building the company's next product: an agent engine, currently in public beta. His thesis is that agents don't fail because models are bad. They fail because the data feeding them is wrong: context poisoning is killing them.

Michel demos this live. A simple Gong query through raw API calls burned 30,000 extra tokens and took three minutes. The same query through Airbyte's context store ran in one minute and used a fraction of the context window. Conor and Michel dig into why RAG alone won't cut it, what a "context engineer" actually does, how Airbyte tracks entities across Salesforce, Zendesk, and Gong without embeddings, and whether the SaaS apocalypse playing out in public markets is overblown.

Chapters:

0:00 Intro
0:20 Meet Michel Tricot, CEO of Airbyte
2:27 Data Got Us to the Information Age. Context Gets Us to Intelligence.
4:48 How Context Poisoning Breaks Agents
7:49 Why Airbyte Customers Stopped Loading Into Warehouses
10:12 Live Demo: Context Store vs Raw API Calls
10:38 What Does a Context Engineer Actually Do?
14:14 RAG Isn't Dead, But How We Build It Will Die
16:41 30K Wasted Tokens Without Proper Context
22:22 Cross-System Joins: Zendesk, Gong, and Salesforce
26:12 The Open Source Agent Connector SDK
29:45 The SaaS Apocalypse Is Overblown
36:09 From Data Pipes to Agent Infrastructure
38:51 What Agents Need to Get Right by Summer
40:48 Memory Is Just Another Form of Context
43:07 Outro

About the Guest:

Michel Tricot is the CEO and co-founder of Airbyte, the open source data integration platform used by thousands of companies to move data between systems. Before Airbyte, he led data ingestion and distribution engineering at LiveRamp. Airbyte raised at a $1.5 billion valuation and offers 600+ free connectors. The company recently launched the public beta of its agent engine, which includes a context store, agent connector SDK, and MCP integration.

Guest Links:

Show Links:

Thanks to our presenting sponsor Galileo. Download their free 165-page guide to mastering multi-agent systems at galileo.ai.

I Started r/AI_Agents and Now I'm Launching a VC Fund

Tue, 10 Mar 2026 13:00:01 GMT

Yujian Tang started the r/AI_Agents subreddit in April 2023. For the first year, it barely moved. Then it hit 9,000 members, he went on vacation, came back to 36,000, and now it's approaching 300,000. In this episode, Yujian talks about how that community grew alongside his event business (Seattle Startup Summit, 900+ attendees last year), his two failed startups, and why he just filed paperwork to launch his own venture fund.Conor and Yujian dig into the mechanics of starting a fund from scratch (Delaware PO boxes, EIN numbers, lawyers), why AI startup valuations have doubled in the last two years, whether a one-person unicorn is realistic, and what failed founders learn that successful ones sometimes miss.Chapters:(0:00) Cold Open: The Subreddit Growth Explosion(0:21) Intro and Meet Yujian Tang(1:06) From AI Research to Community Building(7:26) Where AI Applications Are Headed(10:03) The AI Bubble and a Valuation Reset(10:39) Getting Deal Flow Through Community Events(14:02) Filing the Fund: The Boring Side of VC(16:04) How r/AI_Agents Went from Crickets to 300K(18:39) Building an Accidental Empire(26:37) What Two Failed Startups Taught Him(29:52) Why Pre-Seed Valuations Are Out of Control(37:37) The One-Person Unicorn Debate(39:50) Seattle Startup Summit 2026(42:17) What Chain of Thought Should Cover Next(43:25) OutroAbout the Guest:Yujian Tang is the founder of Seattle Startup Summit, the largest startup event in the Pacific Northwest. He created the r/AI_Agents subreddit (now nearly 300K members), runs hackathons and developer events across Seattle and the Bay Area, and is launching an early-stage AI venture fund.Guest Links:Seattle Startup Summit: seattlestartupsummit.comReddit: reddit.com/r/AI_AgentsShow Links:Chain of Thought Podcast: https://chainofthought.showNewsletter: https://newsletter.chainofthought.show/LinkedIn: https://www.linkedin.com/in/conorbronsdon/X/Twitter: https://x.com/ConorBronsdonSponsor: Thanks to Galileo. Download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

I Built an AI Coworker That Runs 90% of My Day

Wed, 04 Mar 2026 11:00:00 GMT

Sterling Chin stopped thinking of AI as a tool and started treating it like a junior employee. Onboarded it with context, corrected its mistakes, and gave it writing rules. Forty days later, MARVIN was handling 90% of his workday.In this episode of Chain of Thought, Sterling (Applied AI Engineer and Senior Developer Advocate at Postman) walks through live demos of MARVIN, his personal AI assistant built on Claude Code. From pulling meeting transcripts and updating Jira tickets to drafting blog posts and managing his calendar, MARVIN runs as a full-time AI chief of staff.We cover:How MARVIN bookends Sterling's workday from first login to the end of the dayPersonality, sub-agents, and writing rules that make MARVIN an effective co-workerAutomating meeting notes to Jira ticketsWhy DIY assistants outperform big tech alternativesHow Sterling onboarded 12+ colleagues at Postman, including non-technical knowledge workersWhat the compute crunch means for open source AIConnect with Sterling:LinkedIn: https://www.linkedin.com/in/sterlingchin/Twitter/X: https://x.com/SilverJaw82MARVIN Template: https://github.com/SterlingChin/marvin-templateConnect with Conor:Newsletter:⁠ ⁠https://conorbronsdon.substack.com/Twitter/X:⁠ https://x.com/ConorBronsdon⁠LinkedIn:⁠ https://www.linkedin.com/in/conorbronsdonYouTube:⁠⁠ https://www.youtube.com/@ConorBronsdon⁠⁠🔗 More episodes:⁠⁠ https://chainofthought.show⁠⁠Timestamps:(0:00) Intro(0:28) Meet Sterling Chin and the MARVIN AI Assistant(9:10) Live Demo: How MARVIN Bookends Your Workday(16:04) Personality, Sub-Agents, and Writing Rules(22:00) Automating Meeting Notes to Jira Tickets(29:30) Why DIY AI Assistants Outperform Big Tech(40:55) Treat Your AI Like a Junior Employee(46:41) How to Get Started with MARVIN(55:36) The Compute Crunch and Open Source FutureThanks to Galileo — download their free 165-page guide to mastering multi-agent systems at galileo.ai/mastering-multi-agent-systems

How Intercom Cut $250K/Month by Ditching GPT for Qwen

Thu, 26 Feb 2026 10:00:00 GMT

Intercom was spending $250K/month on a single summarization task using GPT. Then they replaced it with a fine-tuned 14B parameter Qwen model and saved almost all of it. In this episode, Intercom's Chief AI Officer, Fergal Reid, walks through exactly how they made that call, where their approach has changed over time, and how all of their efforts built their Fin customer service agent. Fergal breaks down how Fin went from 30% to nearly 70% resolution rate and why most of those gains came from surrounding systems (custom re-rankers, retrieval models, query canonicalization), not the core frontier LLM. He explains why higher latency counterintuitively increases resolution rates, how they built a custom re-ranker that outperformed Cohere using ModernBERT, and why he believes vertically integrated AI products will win in the long term.If you're deciding between fine-tuning open-weight models and using frontier APIs in production, you won't find a more detailed decision process walkthrough.🔗 Connect with Fergal: Twitter/X: https://x.com/fergal_reidLinkedIn: https://www.linkedin.com/in/fergalreid/Fin: https://fin.ai/🔗 Connect with Conor:YouTube: https://www.youtube.com/@ConorBronsdonNewsletter: https://conorbronsdon.substack.com/Twitter/X: https://x.com/ConorBronsdonLinkedIn: https://www.linkedin.com/in/conorbronsdon/🔗 More episodes: https://chainofthought.showCHAPTERS0:00 Intro0:46 Why Intercom Completely Reversed Their Fine-Tuning Position8:00 The $250K/Month Summarization Task (Query Canonicalization)11:25 Training Infrastructure: H200s, LoRA to Full SFT, and GRPO14:09 Why Qwen Models Specifically Work for Production18:03 Goodhart's Law: When Benchmarks Lie19:47 A/B Testing AI in Production: Soft vs. Hard Resolutions25:09 The Latency Paradox: Why Slower Responses Get More Resolutions26:33 Why Per-Customer Prompt Branching Is Technical Debt28:51 Sponsor: Galileo29:36 Hiring Scientists, Not Just Engineers32:15 Context Engineering: Intercom's Full RAG Pipeline35:35 Customer Agent, Voice, and What's Next for Fin39:30 Vertical Integration: Can App Companies Outrun the Labs?47:45 When Engineers Laughed at Claude Code52:23 Closing ThoughtsTAGSFergal Reid, Intercom, Fin AI agent, open-weight models, Qwen models, fine-tuning LLMs, post-training, RAG pipeline, customer service AI, GRPO reinforcement learning, A/B testing AI, Claude Code, vertical AI integration, inference cost optimization, context engineering, AI agents, ModernBERT reranker, scaling AI teams, Conor Bronsdon, Chain of Thought

How Block Deployed AI Agents to 12,000 Employees in 8 Weeks w/ MCP | Angie Jones

Wed, 21 Jan 2026 12:00:00 GMT

How do you deploy AI agents to 12,000 employees in just 8 weeks? How do you do it safely? Angie Jones, VP of Engineering for AI Tools and Enablement at Block, joins the show to share exactly how her team pulled it off.Block (the company behind Square and Cash App) became an early adopter of Model Context Protocol (MCP) and built Goose, their open-source AI agent that's now a reference implementation for the Agentic AI Foundation. Angie shares the challenges they faced, the security guardrails they built, and why letting employees choose their own models was critical to adoption.We also dive into vibe coding (including Angie's experience watching Jack Dorsey vibe code a feature in 2 hours), how non-engineers are building their own tools, and what MCP unlocks when you connect multiple systems together.Chapters:00:00 Introduction02:02 How Block deployed AI agents to 12,000 employees05:04 Challenges with MCP adoption and security at scale07:10 Why Block supports multiple AI models (Claude, GPT, Gemini)08:40 Open source models and local LLM usage09:58 Measuring velocity gains across the organization10:49 Vibe coding: Benefits, risks & Jack Dorsey's 2-hour feature build13:46 Block's contributions to the MCP protocol14:38 MCP in action: Incident management + GitHub workflow demo15:52 Addressing MCP criticism and security concerns18:41 The Agentic AI Foundation announcement (Block, Anthropic, OpenAI, Google, Microsoft)21:46 AI democratization: Non-engineers building MCP servers24:11 How to get started with MCP and prompting tips25:42 Security guardrails for enterprise AI deployment29:25 Tool annotations and human-in-the-loop controls30:22 OAuth and authentication in Goose32:11 Use cases: Engineering, data analysis, fraud detection35:22 Goose in Slack: Bug detection and PR creation in 5 minutes38:05 Goose vs Claude Code: Open source, model-agnostic philosophy38:17 Live Demo: Council of Minds MCP server (9-persona debate)45:52 What's next for Goose: IDE support, ACP, and the $100K contributor grant47:57 Where to get started with GooseConnect with Angie on LinkedIn: https://www.linkedin.com/in/angiejones/Angie's Website: https://angiejones.tech/Follow Angie on X: https://x.com/techgirl1908Goose GitHub: https://github.com/block/gooseConnect with Conor on LinkedIn: https://www.linkedin.com/in/conorbronsdon/Follow Conor on X: https://x.com/conorbronsdonModular: https://www.modular.com/Presented By: Galileo AIDownload Galileo's Mastering Multi-Agent Systems for free here: https://galileo.ai/mastering-multi-agent-systemsTopics Covered:- How Block deployed Goose to all 12,000 employees- Building enterprise security guardrails for AI agents- Model Context Protocol (MCP) deep dive- Vibe coding benefits and risks- The Agentic AI Foundation (Block, Anthropic, OpenAI, Google, Microsoft, AWS)- MCP sampling and the Council of Minds demo- OAuth authentication for MCP servers- Goose vs Claude Code and other AI coding tools- Non-engineers building AI tools- Fraud detection with AI agents- Goose in Slack for real-time bug fixing

Gemini 3 & Robot Dogs: Inside Google DeepMind's AI Experiments | Paige Bailey

Wed, 14 Jan 2026 16:30:00 GMT

Google DeepMind is reshaping the AI landscape with an unprecedented wave of releases—from Gemini 3 to robotics and even data centers in space. Paige Bailey, AI Developer Relations Lead at Google DeepMind, joins us to break down the full Google AI ecosystem. From her unique journey as a geophysicist-turned-AI-leader who helped ship GitHub Copilot, to now running developer experience for DeepMind's entire platform, Paige offers an insider's view of how Google is thinking about the future of AI.The conversation covers the practical differences between Gemini 3 Pro and Flash, when to use the open-source Gemma models, and how tools like Anti-Gravity IDE, Jules, and Gemini CLI fit into developer workflows. Paige also demonstrates Space Math Academy—a gamified NASA curriculum she built using AI Studio, Colab, and Anti-Gravity—showing how modern AI tools enable rapid prototyping. The discussion then ventures into AI's physical frontier: robotics powered by Gemini on Raspberry Pi, Google's robotics trusted tester program, and the ambitious Project Suncatcher exploring data centers in space.00:00 Introduction01:30 Paige's Background & Connection to Modular02:29 Gemini Integration Across Google Products03:04 Jules, Gemini CLI & Anti-Gravity IDE Overview03:48 Gemini 3 Flash vs Pro: Live Demo & Pricing06:10 Choosing the Right Gemini Model09:42 Google's Hardware Advantage: TPUs & JAX10:16 TensorFlow History & Evolution to JAX11:45 NeurIPS 2025 & Google's Research Culture14:40 Google Brain to DeepMind: The Merger Story15:24 Palm II to Gemini: Scaling from 40 People18:42 Gemma Open Source Models20:46 Anti-Gravity IDE Deep Dive23:53 MCP Protocol & Chrome DevTools Integration26:57 Gemini CLI in Google Colab28:00 Image Generation & AI Studio Traffic Spikes28:46 Space Math Academy: Gamified NASA Curriculum31:31 Vibe Coding: Building with AI Studio & Anti-Gravity36:02 AI From Bits to Atoms: The Robotics Frontier36:40 Stanford Puppers: Gemini on Raspberry Pi Robots38:35 Google's Robotics Trusted Tester Program40:59 AI in Scientific Research & Automation42:25 Project Suncatcher: Data Centers in Space45:00 Sustainable AI Infrastructure47:14 Non-Dystopian Sci-Fi Futures47:48 Closing Thoughts & Resources- Connect with Paige on LinkedIn: https://www.linkedin.com/in/dynamicwebpaige/- Follow Paige on X: https://x.com/DynamicWebPaige- Paige's Website: https://webpaige.dev/- Google DeepMind: https://deepmind.google/- AI Studio: https://ai.google.devConnect with our host Conor Bronsdon:- Substack – https://conorbronsdon.substack.com/ - LinkedIn https://www.linkedin.com/in/conorbronsdon/Presented By: Galileo.aiDownload Galileo's Mastering Multi-Agent Systems for free here!: https://galileo.ai/mastering-multi-agent-systemsTopics Covered:- Gemini 3 Pro vs Flash comparison (pricing, speed, capabilities)- When to use Gemma open-source models- Anti-Gravity IDE, Jules, and Gemini CLI workflows- Google's TPU hardware advantage- History of TensorFlow, JAX, and Google Brain- Space Math Academy demo (gamified education)- AI-powered robotics (Stanford Puppers on Raspberry Pi)- Project Suncatcher (orbital data centers)

Explaining Eval Engineering | Galileo's Vikram Chatterji

Fri, 19 Dec 2025 10:00:00 GMT

You've heard of evaluations—but eval engineering is the difference between AI that ships and AI that's stuck in prototype.Most teams still treat evals like unit tests: write them once, check a box, move on. But when you're deploying agents that make real decisions, touch real customers, and cost real money, those one-time tests don't cut it. The companies actually shipping production AI at scale have figured out something different—they've turned evaluations into infrastructure, into IP, into the layer where domain expertise becomes executable governance.Vikram Chatterji, CEO and Co-founder of Galileo, returns to Chain of Thought to break down eval engineering: what it is, why it's becoming a dedicated discipline, and what it takes to actually make it work. Vikram shares why generic evals are plateauing, how continuous learning loops drive accuracy, and why he predicts "eval engineer" will become as common a role as "prompt engineer" once was.In this conversation, Conor and Vikram explore:Why treating evals as infrastructure—not checkboxes—separates production AI from prototypesThe plateau problem: why generic LLM-as-a-judge metrics can't break 90% accuracyHow continuous human feedback loops improve eval precision over timeThe emerging "eval engineer" role and what the job actually looks likeWhy 60-70% of AI engineers' time is already spent on evalsWhat multi-agent systems mean for the future of evaluationVikram's framework for baking trust AND control into agentic applicationsPlus: Conor shares news about his move to Modular and what it means for Chain of Thought going forward.Chapters:00:00 – Introduction: Why Evals Are Becoming IP01:37 – What Is Eval Engineering?04:24 – The Eval Engineering Course for Developers05:24 – Generic Evals Are Plateauing08:21 – Continuous Learning and Human Feedback11:01 – Human Feedback Loops and Eval Calibration13:37 – The Emerging Eval Engineer Role16:15 – What Production AI Teams Actually Spend Time On18:52 – Customer Impact and Lessons Learned24:28 – Multi-Agent Systems and the Future of Evals30:27 – MCP, A2A Protocols, and Agent Authentication33:23 – The Eval Engineer Role: Product-Minded + Technical34:53 – Final Thoughts: Trust, Control, and What's NextConnect with Conor Bronsdon:Substack – https://conorbronsdon.substack.com/LinkedIn – https://www.linkedin.com/in/conorbronsdon/X (Twitter) – https://x.com/ConorBronsdonLearn more about Eval Engineering:⁠https://galileo.ai/evalengineering⁠Connect with Vikram Chatterji:LinkedIn – ⁠https://www.linkedin.com/in/vikram-chatterji/⁠

Debunking AI's Environmental Panic | Andy Masley

Wed, 26 Nov 2025 10:32:57 GMT

AI is destroying the planet—or so we've been told. This week on Chain of Thought, we tackle one of the most persistent and misleading narratives in the AI conversation.Andy Masley, Director of Effective Altruism DC, joins host Conor Bronsdon to fact-check the absurd AI environmental claims you've heard at parties, in articles, and even in bestselling books. Andy recently went viral for discovering what he calls "the single most egregious math mistake" he's ever seen in a book—a data center water usage calculation in Karen Hao's NYT Bestseller, Empire of AI, that was off by a factor of 4,500.In this conversation, Andy and Conor break down the myths around AI’s water and energy usage and explore:The viral Empire of AI error and what it reveals about the broader debateWhy most AI water usage statistics are misleading or flat-out wrongHow one ChatGPT prompt represents just 1/150,000th of your daily emissionsTrade-offs around data center cooling + decision makingWhy "tribal thinking" about AI is distorting environmental activismWhere AI might actually help the climate through deep learning optimizationIf you've ever felt guilty about using AI tools, been cornered at a party about AI's environmental impact, or simply want to understand what the data actually says, this episode, and Andy’s deep dive articles, arm you with the facts.Chapters:00:00 – Introduction: The Party Guilt Problem01:54 – Andy's Background and What Sparked This Work03:50 – The 4,500x Error in Empire of AI06:39 – Breaking Down the Math: Liters vs. Cubic Meters10:39 – The Unintended Consequence: Air Cooling vs. Water Cooling12:51 – Karen Hao's Response and What's Still Missing19:08 – Why Environmentalists Should Focus Elsewhere21:41 – The Danger of Tribal Thinking About AI25:49 – What Is Effective Altruism (And Why People Attack It)29:15 – EA, AI Risk, and P(doom)34:31 – Why Misinformation Hurts Your Own Side37:39 – Using ChatGPT Is Not Bad for the Environment42:14 – The Party Rebuttal: Practical Comparisons45:23 – Water Use Reality: 1/800,000th of Your Daily Footprint48:27 – The Personal Carbon Footprint Distraction53:38 – Data Centers: Efficiency vs. Whether to Build55:13 – AI's Net Climate Impact: The Positive Case59:34 – Deep Learning, Smart Grids, and Climate Optimization1:03:45 – Final ThoughtsKey referencesIEA Study: AI and climate change - https://www.iea.org/reports/energy-and-ai/ai-and-climate-change#abstract Nature: https://www.nature.com/articles/s44168-025-00252-3 The Empire of AI Error: https://andymasley.substack.com/p/empire-of-ai-is-wildly-misleading Using ChatGPT isn’t bad for the environment: https://andymasley.substack.com/p/a-short-summary-of-my-argument-thathttps://andymasley.substack.com/p/a-cheat-sheet-for-conversations-about Connect with Andy Masley: Substack – https://andymasley.substack.com/X (Twitter) – https://x.com/AndyMasleyConnect with Conor Bronsdon: Substack – https://conorbronsdon.substack.com/LinkedIn – https://www.linkedin.com/in/conorbronsdon/X (Twitter) – https://x.com/ConorBronsdon

The Critical Infrastructure Behind the AI Boom | Cisco CPO Jeetu Patel

Wed, 19 Nov 2025 14:00:00 GMT

AI is accelerating at a breakneck pace, but model quality isn’t the only constraint we face.. There are major infrastructure requirements, energy needs, security, and data pipelines to run AI at scale. This week on Chain of Thought, Cisco’s President and Chief Product Officer Jeetu Patel joins host Conor Bronsdon to reveal what it actually takes to build the critical foundation for the AI era.Jeetu breaks down the three bottlenecks he sees holding AI back today: • Infrastructure limits: not enough power, compute, or data center capacity • A trust deficit: non-deterministic models powering systems that must be predictable • A widening data gap: human-generated data plateauing while machine data explodesJeetu then shares how Cisco is tackling these challenges through secure AI factories, edge inference, open multi-model architectures, and global partnerships with Nvidia, G42, and sovereign cloud providers. Jeetu also explains why he thinks enterprises will soon rely on thousands of specialized models — not just one — and how routing, latency, cost, and security shape this new landscape.Conor and Jeetu also explore high-performance leadership and team culture, discussing building high-trust teams, embracing constructive tension, staying vigilant in moments of success, and the personal experiences that shaped Jeetu’s approach to innovation and resilience.If you want a clearer picture of the global AI infrastructure race, how high-level leaders are thinking about the future, and what it all means for enterprises, developers, and the future of work, this conversation is essential.Chapters:00:00 – Welcome to Chain of Thought0:48 - AI and Jobs: Beyond the Hype6:15 - The Real AI Opportunity: Original Insights10:00 - Three Critical AI Constraints: Infrastructure, Trust, and Data16:27 - Cisco's AI Strategy and Platform Approach19:18 - Edge Computing and Model Innovation22:06 - Strategic Partnerships: Nvidia, G42, and the Middle East29:18 - Acquisition Strategy: Platform Over Products32:03 - Power and Infrastructure Challenges36:06 - Building Trust Across Global Partnerships38:03 - US vs. China: The AI Infrastructure Race40:33 - America's Venture Capital Advantage42:06 - Acquisition Philosophy: Strategy First45:45 - Defining Cisco's True North48:06 - Mission-Driven Innovation Culture50:15 - Hiring for Hunger, Curiosity, and Clarity56:27 - The Power of Constructive Conflict1:00:00 - Career Lessons: Continuous Learning1:02:24 - The Email Question1:04:12 - Joe Tucci's Four-Column Exercise1:08:15 - Building High-Trust Teams1:10:12 - The Five Dysfunctions Framework1:12:09 - Leading with Vulnerability1:16:18 - Closing Thoughts and Where to ConnectConnect with Jeetu Patel:LinkedIn – https://www.linkedin.com/in/jeetupatel/ X(twitter) – https://x.com/jpatel41Cisco - https://www.cisco.com/Connect with ConorBronsdon Substack – https://conorbronsdon.substack.com/ LinkedIn – https://www.linkedin.com/in/conorbronsdon/X (twitter) – https://x.com/ConorBronsdon

Beyond Transformers: How Liquid AI Is Rethinking LLM Architecture | Maxime Labonne

Wed, 12 Nov 2025 10:00:00 GMT

The transformer architecture has dominated AI since 2017, but it’s not the only approach to building LLMs - and new architectures are bringing LLMs to edge devicesMaxime Labonne, Head of Post-Training at Liquid AI and creator of the 67,000+ star LLM Course, joins Conor Bronsdon to challenge the AI architecture status quo. Liquid AI’s hybrid architecture, combining transformers with convolutional layers, delivers faster inference, lower latency, and dramatically smaller footprints without sacrificing capability. This alternative architectural philosophy creates models that run effectively on phones and laptops without compromise.But reimagined architecture is only half the story. Maxime unpacks the post-training reality most teams struggle with: challenges and opportunities of synthetic data, how to balance helpfulness against safety, Liquid AI’s approach to evals, RAG architectural approaches, how he sees AI on edge devices evolving, hard won lessons from shipping LFM1 through 2, and much more. If you're tired of surface-level AI takes and want to understand the architectural and engineering decisions behind production LLMs from someone building them in the trenches, this is your episode.Connect with ⁨Maxime Labonne⁩ :LinkedIn – https://www.linkedin.com/in/maxime-labonne/ X (Twitter) – @maximelabonneAbout Maxime – https://mlabonne.github.io/blog/about.html HuggingFace – https://huggingface.co/mlabonne The LLM Course – https://github.com/mlabonne/llm-course Liquid AI – https://liquid.ai Connect with ⁨Conor Bronsdon⁩ :X (twitter) – @conorbronsdonSubstack – https://conorbronsdon.substack.com/ LinkedIn – https://www.linkedin.com/in/conorbronsdon/00:00 Intro — Welcome to Chain of Thought 00:27 Guest Intro — Maxime Labonne of Liquid AI 02:21 The Hybrid LLM Architecture Explained 06:30 Why Bigger Models Aren’t Always Better 11:10 Convolution + Transformers: A New Approach to Efficiency 18:00 Running LLMs on Laptops and Wearables 22:20 Post-Training as the Real Moat 25:45 Synthetic Data and Reliability in Model Refinement 32:30 Evaluating AI in the Real World 38:11 Benchmarks vs Functional Evals 43:05 The Future of Edge-Native Intelligence 48:10 Closing Thoughts & Where to Find Maxime Online

Architecting AI Agents: The Shift from Models to Systems | Aishwarya Srinivasan

Wed, 08 Oct 2025 13:00:00 GMT

Most AI agents are built backwards, starting with models instead of system architecture.Aishwarya Srinivasan, Head of AI Developer Relations at Fireworks AI, joins host Conor Bronsdon to explain the shift required to build reliable agents: stop treating them as model problems and start architecting them as complete software systems. Benchmarks alone won't save you. Aish breaks down the evolution from prompt engineering to context engineering, revealing how production agents demand careful orchestration of multiple models, memory systems, and tool calls. She shares battle-tested insights on evaluation-driven development, the rise of open source models like DeepSeek v3, and practical strategies for managing autonomy with human-in-the-loop systems. The conversation addresses critical production challenges, ranging from LLM-as-judge techniques to navigating compliance in regulated environments.Connect with Aishwarya Srinivasan:LinkedIn: https://www.linkedin.com/in/aishwarya-srinivasan/Instagram: https://www.instagram.com/the.datascience.gal/Connect with Conor: https://www.linkedin.com/in/conorbronsdon/00:00 Intro — Welcome to Chain of Thought00:22 Guest Intro — Ash Srinivasan of Fireworks AI02:37 The Challenge of Responsible AI05:44 The Hidden Risks of Reward Hacking07:22 From Prompt to Context Engineering10:14 Data Quality and Human Feedback14:43 Quantifying Trust and Observability20:27 Evaluation-Driven Development30:10 Open Source Models vs. Proprietary Systems34:56 Gaps in the Open-Source AI Stack38:45 When to Use Different Models45:36 Governance and Compliance in AI Systems50:11 The Future of AI Builders56:00 Closing Thoughts & Follow Ash OnlineFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

The Accidental Algorithm | Humans of AI Crossover with Writer's Melisa Russak

Wed, 01 Oct 2025 14:00:00 GMT

This week, we're doing something special and sharing an episode from another podcast we love: The Humans of AI by our friends at Writer. We're huge fans of their work, and you might remember Writer's CEO, May Habib, from the inaugural episode of our own show.From The Humans of AI:Learn how Melisa Russak, lead research scientist at WRITER, stumbled upon fundamental machine learning algorithms, completely unaware of existing research — twice. Her story reveals the power of approaching problems with fresh eyes and the innovative breakthroughs that can occur when constraints become catalysts for creativity.Melisa explores the intersection of curiosity-driven research, accidental discovery, and systematic innovation, offering valuable insights into how WRITER is pushing the boundaries of enterprise AI. Tune in to learn how her journey from a math teacher in China to a pioneer in AI research illuminates the future of technological advancement.Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Check out Writer’s YouTube channel to watch the full interviews. Learn more about WRITER at writer.com. Follow Melisa on LinkedInFollow May on LinkedInCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

After Code Gen: What Graphite Is Building for the Post-AI Dev Stack | Greg Foster

Wed, 24 Sep 2025 14:00:00 GMT

The incredible velocity of AI coding tools has shifted the critical bottleneck in software development from code generation to code reviews. Greg Foster, Co-Founder & CTO of Graphite, joins the conversation to explore this new reality, outlining the three waves of AI that are leading to autonomous agents spawning pull requests in the background. He argues that as AI automates the "inner loop" of writing code, the human-centric "outer loop"—reviewing, merging, and deploying—is now under immense pressure, demanding a complete rethinking of our tools and processes.The conversation then gets tactical, with Greg detailing how a technique called "stacking" can break down large code changes into manageable units for both humans and AI. He also identifies an emerging hiring gap where experienced engineers with strong architectural context are becoming "lethal" with AI tools. This episode is an essential guide to navigating the new bottlenecks in software development and understanding the skills that will define the next generation of high-impact engineers.Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Greg on LinkedInFollow Greg on XGraphite Website: graphite.devCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Vercel's Playbook for AI Agents: From Vibe Check to Production | Malte Ubl

Wed, 10 Sep 2025 13:00:00 GMT

What’s the first step to building an enterprise-grade AI tool? Malte Ubl, CTO of Vercel, joins us this week to share Vercel’s playbook for agents, explaining how agents are a new type of software for solving flexible tasks. He shares how Vercel's developer-first ecosystem, including tools like the AI SDK and AI Gateway, is designed to help teams move from a quick proof-of-concept to a trusted, production-ready application.Malte explores the practicalities of production AI, from the importance of eval-driven development to debugging chaotic agents with robust tracing. He offers a critical lesson on security, explaining why prompt injection requires a totally different solution - tool constraint - than traditional threats like SQL injection. This episode is a deep dive into the infrastructure and mindset, from sandboxes to specialized SLMs, required to build the next generation of AI tools.Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Malte on LinkedInFollow Malte on X (formerly Twitter)Learn more about VercelCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

From Demo to Defensibility: How to Build an AI Business that Lasts | Aurimas Griciūnas

Wed, 27 Aug 2025 14:00:00 GMT

The technological moat is eroding in the AI era, what new factors separate a successful startup from the rest?Aurimas Griciūnas, CEO of SwirlAI, joins the show to break down the realities of building in this new landscape. Startup success now hinges on speed, strong financial backing, or immediate distribution. Aurimas warns against the critical mistake of prioritizing shiny tools over fundamental engineering and the market gaps this creates.Discover the new moats for AI companies, built on a culture of relentless execution, tight feedback loops, and the surprising skills that define today's most valuable engineers.The episode also looks to the future, with bold predictions about a slowdown in LLM leaps and the coming impact of coding agents and self-improving systems.Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Aurimas on⁠ ⁠⁠LinkedIn⁠Aurimas' Course: ⁠End-to-End AI Engineering BootcampCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Mindset Over Metrics: How to Approach AI Engineering | Hamel Husain

Wed, 20 Aug 2025 13:00:00 GMT

As we enter the era of the AI engineer, the biggest challenge isn't technical - it's a shift in mindset. Hamel Husain, a leading AI consultant and luminary in the eval space, joins the podcast to explore the skills and processes needed to build reliable AI. Hamel explains why many teams relying on vanity dashboards and a "buffet of metrics" experience a false sense of security, which is no substitute for customized evals tailored to domain-specific risks. The solution? A disciplined process of error analysis, grounded in manually looking at data to identify real-world failures This discussion is an essential guide to building the continuous learning loops and "experimentation mindset" required to take AI products from prototype to production with confidence. Listen to learn the playbook for building AI reliability, and derive qualitative insights from log data to build customized quantitative guardrails. Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Hamel on LinkedInFollow Hamel on X/TwitterCheck out his blog: hamel.devCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

How AI Velocity is Rewriting the Rules for Engineering Leaders | ChatPRD's Claire Vo

Wed, 13 Aug 2025 13:00:00 GMT

What if your next competitor is not a startup, but a solo builder on a side project shipping features faster than your entire team? For Claire Vo, that's not a hypothetical. As the founder of ChatPRD, formerly the Chief Product and Technology Officer at LaunchDarkly, and host of the How I AI podcast, she has a unique vantage point on the driving forces behind a new blueprint for success.She argues that AI accountability must be driven from the top by an "AI czar" and reveals how a culture of experimentation is the key to overcoming organizational hesitancy. Drawing from her experience as a solo founder, she warns that for incumbents, the cost of moving slowly is the biggest threat and details how AI can finally be used to tackle legacy codebases. The conversation closes with bold predictions on the rise of the "super IC" - who can achieve top-tier impact and salary without managing a team - and the death of product management. Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Claire on LinkedInFollow Claire on X/TwitterClaire’s podcast How I AICheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Building an AI-Native Startup | GrowthX's Marcel Santilli

Wed, 06 Aug 2025 14:00:00 GMT

How do you build an AI-native company to a $7M run rate in just six months?According to Marcel Santilli, Founder and CEO of GrowthX, the secret isn't chasing the next frontier model, it's mastering the "messy middle." Drawing on his deep experience at Scale AI and Deepgram, Marcel joins host Conor Bronsdon to share his framework for building durable, customer-obsessed businesses.Marcel argues that the most critical skills for the AI era aren't technical but philosophical: first-principles thinking and the art of delegation.Tune in to learn why GrowthX first focused on services to codify expert work, how AI can augment human talent instead of replacing it, and why speed and brand are a startup's greatest competitive advantages. This conversation offers a clear playbook for building a resilient company by prioritizing culture and relentless shipping.Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Marcel on LinkedInFollow Marcel on X (formerly Twitter)Learn more about GrowthXCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

AI's Trillion-Dollar Healthcare Bet | Corti's Andreas Cleve

Wed, 30 Jul 2025 14:00:00 GMT

AI isn't just changing healthcare; it's providing the essential help needed to unlock a trillion-dollar opportunity for better care.Andreas Cleve, CEO & Co-founder of Corti, steps in to shed light on AI's immense, yet often misunderstood, transformative potential in this high-stakes environment. Andreas refutes the narrative of healthcare being slow adopters, emphasizing its high bar for trustworthy technology and its constant embrace of new tools. He reveals how purpose-built AI models are already alleviating the "pajama time" burden of documentation for clinicians, enabling faster and more accurate assessments in various specializations. This quiet, impactful adoption is seeing companies grow "like weeds" beyond common expectations.The conversation addresses how AI can tackle the looming global shortage of 10 million healthcare professionals by 2030, reallocating a trillion dollars worth of administrative work back into care. Andreas details Corti’s approach to building invisible, reliable AI through rigorous, compliance-first evaluation, ensuring accuracy and efficiency in real-time. He emphasizes that AI's true role is not replacement, but augmentation, empowering professionals to deliver more care, attract talent, and drive organizational growth.Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)LinkedIn: linkedin.com/in/andreascleveX (formerly Twitter): andreascleveCorti Website: corti.aiCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Mastering Multi-Agent Systems | MongoDB’s Mikiko Chandrasekhar

Wed, 23 Jul 2025 13:00:00 GMT

AI agents offer unprecedented power, but mastering agent reliability is the ultimate challenge for agentic systems to actually work in production.Mikiko Chandrashekar, Staff Developer Advocate at MongoDB, whose background spans the entire data-to-AI pipeline, unveils MongoDB's vision as the memory store for agents, supporting complex multi-agent systems from data storage and vector search to debugging chat logs. She highlights how MongoDB, reinforced by the acquisition of Voyage, empowers developers to build production-scale agents across various industries, from solo projects to major enterprises. This robust data layer is foundational to ensure agent performance and improve the end user experience.Mikiko advocates for treating agents as software products, applying rigorous engineering best practices to ensure reliability, even for non-deterministic systems. She details MongoDB's unique position to balance GPU/CPU loads and manage data for performance and observability, including Galileo's integrations. The conversation emphasizes the profound need to rethink observability, evaluations, and guardrails in the era of agents, showcasing Galileo's family of small language models for real-time guardrailing, Luna-2, and Insights Engine for automated failure analysis. Discover how building trustworthiness through systematic evaluation, beyond just "vibe checks," is essential for AI agents to scale and deliver value in high-stakes use cases.Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Mikiko on LinkedInFollow Mikiko on X/TwitterExplore Mikiko's YouTube channelCheck out Mikiko's ⁠SubstackConnect with MongoDB on LinkedInConnect with MongoDB on YouTubeCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

The AI Agent Trust Gap: Bridging Risk to Reliability | Elastic’s Philipp Krenn

Wed, 16 Jul 2025 15:00:00 GMT

The age of ubiquitous AI agents is here, bringing immense potential - and unprecedented risk.Hosts Conor Bronsdon and Vikram Chatterji open the episode by discussing the urgent need for building trust and reliability into next-generation AI agents. Vikram unveils Galileo's free AI reliability platform for agents, featuring Luna 2 SLMs for real-time guardrails and its Insights Engine for automatic failure mode analysis. This platform enables cost-effective, low-latency production evaluations, significantly transforming debugging. Achieving trustworthy AI agents demands rigorous testing, continuous feedback, and robust guardrailing—complex challenges requiring powerful solutions from partners like Elastic.Conor welcomes Philipp Krenn, Director of Developer Relations at Elastic, to discuss their collaboration in ensuring AI agent reliability, including how Elastic leverages Galileo's platform for evaluation. Philipp details Elastic's evolution from a search powerhouse to a key AI enabler, transforming data access with Retrieval-Augmented Generation (RAG) and new interaction modes. He discusses Elastic's investment in SLMs for efficient re-ranking and embeddings, emphasizing robust evaluation and observability for production. This collaborative effort aims to equip developers to build reliable, high-performing AI systems for every enterprise.Chapters:00:00 Introduction 01:09 Galileo's AI Reliability Platform01:43 Challenges in AI Agent Reliability06:17 Insights Engine and Its Importance11:00 Luna 2: Small Language Models14:42 Custom Metrics and Agent Leaderboard19:16 Galileo's Integrations and Partnerships21:04 Philipp Krenn from Elastic24:47 Optimizing LLM Responses 25:41 Galileo and Elastic: A Powerful Partnership28:20 Challenges in AI Production and Trust30:02 Guardrails and Reliability in AI Systems32:17 The Future of AI in Customer InteractionFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Philipp on LinkedInLearn more about ElasticCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Architecting Reliable Agentic AI | Cisco’s Giovanna Carofiglio on the AGNTCY Collective

Wed, 09 Jul 2025 14:00:00 GMT

The Internet of Agents is rapidly taking shape, necessitating innovative foundational standards, protocols, and evaluation methods for its success.Recorded at Cisco's office in San Jose, we welcome Giovanna Carofiglio, Distinguished Engineer and Senior Director at Outshift by Cisco. As a leader of the AGNTCY Collective (an open-source initiative by Cisco, Galileo, LangChain, and many other participating companies), Giovanna outlines the vision for agents to collaborate seamlessly across the enterprise and the internet. She details the collective's pillars, from agent discovery and deployment using new agentic protocols like Slim, to ensuring a secure, low-latency communication transport layer. This groundbreaking work aims to make distributed agentic communication a reality.The conversation then explores the critical role of observability and evaluation in building trustworthy agent applications, including defining an interoperable standard schema for communications. Giovanna highlights the complex challenges of scaling agents to thousands or millions, emphasizing the need for robust security (agent identity with OSF schema) and predictable agent behavior through extensive testing and characterization. She distinguishes between protocols like MCP (agent-to-tool) and A2A (agent-to-agent), advocating for open standards and underlying transport layers akin to TCP. Chapters:00:00 Introduction01:00 Overview of Agent Interoperability02:20 What is AGNTCY03:45 Agent Discovery and Composition04:38 Agent Protocols and Communication05:45 Observability and Evaluation07:00 Metrics and Standards for Agents09:45 Challenges in Agent Evaluation14:15 Low Latency and Active Evaluation23:34 Synthetic Data and Ground Truth25:07 Interoperable Agent Schema26:37 MCP & A2A30:17 Future of Agent Communication32:03 Security and Agent Identity34:37 Collaboration and Community Involvement38:28 Conclusion Follow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)AGNTCY Collective: agntcy.orgConnect with Giovanna on LinkedInLearn more about Outshift: outshift.cisco.comCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Taste Is The New Moat | Why Customer Obsession Wins in the AI Era

Wed, 02 Jul 2025 15:00:00 GMT

When AI makes creating content and code nearly free, how do you stand out? Differentiation now hinges on two things: unique taste and effective distribution.This week, Bharat Vasan, founder & CEO at Intangible and a "recovering VC," explains why the AI landscape compelled him to return to founding. He sees AI sparking a new creative revolution, similar to the early internet, that makes it easier than ever to bring ideas to life. The conversation delivers essential advice for founders, revealing why relentless shipping is the ultimate clarifier for a business and why resilience, not just intelligence, is the key to survival.Drawing from his experience on both sides of the venture table, Bharat breaks down the brutally competitive VC landscape and shares Intangible's mission: to simplify 3D creative tools with AI, finally bridging the gap between human vision and machine power. Listeners will gain insights on company building, brand strategy, and why customer obsession is the ultimate moat in the AI age.Chapters:00:00 Introduction 00:45 From Founder to VC and Back03:17 Human Creativity in the Age of AI07:50 The Role of Taste and Distribution11:49 Building a Brand in the AI Era16:17 The Venture Capital Landscape for AI Startups20:11 Advice for Founders in the AI Boom23:55 Incumbents vs. Startups27:10 The New Generation of Innovators29:19 Pirate Mentality in Startups30:00 Building a Brand36:28 Shipping and Resilience41:49 Customer Obsession46:58 The Vision for Intangible51:52 ConclusionFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Bharat on LinkedIn.Follow Bharat on X.Learn more about Intangible at intangible.ai.Check out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

The Emerging AI Agent Stack | CrewAI’s João Moura

Wed, 25 Jun 2025 10:00:00 GMT

Unlocking AI agents for knowledge work automation and scaling intelligent, multi-agent systems within enterprises fundamentally requires measurability, reliability, and trust.João Moura, founder & CEO of CrewAI, joins Galileo’s Conor Bronsdon and Vikram Chatterji to unpack and define the emerging AI agent stack. They explore how enterprises are moving beyond initial curiosity to tackle critical questions around provisioning, authentication, and measurement for hundreds or thousands of agents in production. The discussion highlights a crucial "gold rush" among middleware providers, all racing to standardize the orchestration and frameworks needed for seamless agent deployment and interoperability. This new era demands a re-evaluation of everything from cloud choices to communication protocols as agents reshape the market.João and Vikram then dive into the complexities of building for non-deterministic multi-agent systems, emphasizing the challenges of increased failure modes and the need for rigorous testing beyond traditional software. They detail how CrewAI is democratizing agent access with a focus on orchestration, while Galileo provides the essential reliability platform, offering advanced evaluation, observability, and automated feedback loops. From specific use cases in financial services to the re-emergence of core data science principles, discover how companies are building trustworthy, high-quality AI products and prepare for the coming agent marketplace. Chapters:00:00 Introduction and Guest Welcome02:04 Defining the AI Agent Stack03:49 Challenges in Building AI Agents05:52 The Future of AI Agent Marketplaces06:59 Infrastructure and Protocols09:05 Interoperability and Flexibility20:18 Governance and Security Concerns24:12 Industry Adoption and Use Cases25:57 Unlocking Faster Development with Success Metrics28:40 Challenges in Managing Complex Systems30:10 Introducing the Insights Engine30:33 The Importance of Observability and Control32:33 Democratizing Access with No-Code Tools35:39 Ensuring Quality and Reliability in Production41:08 Future of Agentic Systems and Industry TransformationFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Joao Moura: LinkedIn | X/TwitterCrewAI: crewai.com | X/Twitter Check out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

AMD's Challenge to NVIDIA: The Open Ecosystem Bet | Anush Elangovan & Sharon Zhou

Wed, 18 Jun 2025 10:00:00 GMT

How is an open ecosystem powering the next generation of AI for developers and leaders?Broadcasting live from the heart of the action at AMD's Advancing AI 2025, Chain of Thought host Conor Bronsdon welcomes AMD’s Anush Elangovan, VP of AI Software, and Sharon Zhou, VP of AI. They unpack AMD's groundbreaking transformation from a hardware giant to a leader in full-stack AI, committed to an open ecosystem. Discover how new MI350 GPUs deliver mind-blowing performance with advanced data types and why ROCm 7 and AMD Developer Cloud offer Day Zero support for frontier models.Then Conor welcomes Sharon Zhou, VP of AI at AMD, to discuss making AMD's powerful software stack truly accessible and how to drive developer curiosity. Sharon explains strategies for creating a "happy path" for community contributions, fostering engagement through teaching, and listening to developers at every stage. She shares her predictions for the future, including the rise of self-improving AI, the critical role of heterogeneous compute, and the potential of "vibes based feedback" to guide models. This vision for democratizing access to high-performance AI, driven by a deep understanding of the developer journey, promises to unlock the next generation of applications.Chapters:00:00 Live from AMD's Advancing AI 2025 Event00:30 Introduction to Anush Elangovan01:38 The MI350 GPU Series Unveiled04:57 CDNA4 Architecture Explained07:00 The Future of AI Infrastructure08:32 AMD's Developer Cloud and ROCm 711:50 Cultural Shift at AMD14:48 Open Source and Community Contributions18:35 Software Longevity and Ecosystem Strategy22:19 AI Agents and Performance Gains27:36 AI's Role in Solving Power Challenges28:11 Thanking Anush28:42 Introduction to Sharon Zhou29:45 Sharon's Focus at AMD30:39 Engaging Developers with AMD's AI Tools31:24 Listening to the AI Community33:56 Open Source and AI Development45:04 Future of AI and Self-Improving Models48:04 Final Thoughts and FarewellFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Anush Elangovan: LinkedInSharon Zhou: LinkedInAMD Official Site: amd.comAMD Developer Resources: AMD Developer CentralCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Your Key to AI Success is Hiding in Plain Sight | Cohesity's Greg Statton

Wed, 11 Jun 2025 10:00:00 GMT

What if the most valuable data in your enterprise—the key to your AI future—is sitting dormant in your backups, treated like an insurance policy you hope to never use?Join Conor Bronsdon with Greg Statton, VP of AI Solutions at Cohesity, for an inside look at how they are turning this passive data into an active asset to power generative AI applications. Greg details Cohesity’s evolution from an infinitely scalable file system built for backups into a data intelligence powerhouse, managing hundreds of exabytes of enterprise data globally. He recounts how early successes in using this data for security and anomaly detection paved the way for more advanced AI applications. This foundational work was crucial in preparing Cohesity to meet the new demands of generative AI.Greg offers a candid look at the real-world challenges enterprises face, arguing that establishing data hygiene and a cross-functional governance model is the most critical step before building reliable AI applications. He shares the compelling story of how Cohesity's focus on generative AI was sparked by an internal RAG experiment he built to solve a "semantic divide" in team communication, which quickly grew into a company-wide initiative. He also provides essential advice for data professionals, emphasizing the need to focus on solving core business problems.Chapters:00:00 Introduction00:36 The Role of Gaming in AI Development05:43 Personal Gaming Experiences08:26 The Intersection of AI and Gaming12:53 Importance of Data in Game Development19:03 User Testing and QA in Gaming25:49 Postmortems and Telemetry27:21 Beta Testing and Data Preparedness29:18 Traditional AI vs Generative AI31:31 Challenges of Implementing AI in Games35:57 Leveraging AI for Data Analytics39:41 Automated QA and Reinforcement Learning42:01 AI for Localization and Sentiment Analysis44:21 Future of AI in GamingFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Company Website: cohesity.comLinkedIn: Gregory StattonCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Why Gamers Paved the Way for AI | Databricks' Carly Taylor

Wed, 04 Jun 2025 10:30:00 GMT

What if the pixels and polygons of your favorite video games were the secret architects of today's AI revolution?Carly Taylor, Field CTO for Gaming at Databricks and founder of ggAI, joins host Conor Bronsdon to illuminate the direct line from video game innovation to the current AI landscape. She explains how the gaming industry's relentless pursuit of better graphics and performance not only drove pivotal GPU advancements and cost reductions, but also fundamentally shaped our popular understanding of artificial intelligence by popularizing the very term "AI" through decades of in-game experiences. Carly shares her personal journey, from a childhood passion for games like Rollercoaster Tycoon ignited while playing with her mom, to becoming a data scientist for Call of Duty. The discussion then confronts a long-standing tension in game development: how the critical need to ship titles often relegates vital game data to a secondary concern, a dynamic Carly explains is now being reshaped by AI. She details the inherent challenges game studios face in capturing and leveraging telemetry, from disparate development processes to the lengthy pipeline required for updates. Carly illuminates how modern AI, particularly generative AI, presents a massive opportunity for studios to finally unlock their vast data troves for everything from self-service analytics and community insight generation to revolutionizing QA processes. This pivotal intersection of evolving game data practices and new AI capabilities is poised to redefine how games are made, understood, and ultimately experienced.Chapters00:00 Introduction00:28 The Role of Gaming in AI Development05:35 Personal Gaming Experiences08:18 The Intersection of AI and Gaming12:45 Importance of Data in Game Development18:55 User Testing and QA in Gaming25:41 Postmortems and Telemetry27:13 Beta Testing and Data Preparedness29:10 Traditional AI vs Generative AI31:23 Challenges of Implementing AI in Games35:49 Leveraging AI for Data Analytics39:33 Automated QA and Reinforcement Learning41:53 AI for Localization and Sentiment Analysis44:13 Future of AI in GamingFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Connect with Carly on LinkedInSubscribe to Carly's Substack: Good At BusinessCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

The 2025 AI Shift: From Chat to Task Completion & Reliable Action | Galileo Founders

Wed, 28 May 2025 10:00:00 GMT

AI in 2025 promises intelligent action, not just smarter chat. But are enterprises prepared for the agentic shift and the complex reliability hurdles it brings?Join Conor Bronsdon on Chain of Thought with fellow co-hosts and Galileo co-founders, Vikram Chatterji (CEO) and Atindriyo Sanyal (CTO), as they explore this pivotal transformation. They discuss how generative AI is evolving from a simple tool into a powerful engine for enterprise task automation, a significant advance driving the pursuit of substantial ROI. This shift is also fueling what Vikram observes as a "gold rush" for middleware and frameworks, alongside healthy skepticism about making widespread agentic task completion a practical reality.As these AI systems grow into highly complex, compound structures—often incorporating multimodal inputs and multi-agent designs—Vikram and Atin address the critical challenges around debugging, achieving reliability, and solving the profound measurement problem. They share Galileo's vision for an AI reliability platform designed to tame these intricate systems through robust guardrailing, advanced metric engines like Luna, and actionable developer insights. Tune in to understand how the industry is moving beyond point-in-time evaluations to continuous AI reliability, crucial for building trustworthy, high-performing AI applications at scale.Chapters00:00 Welcome and Introductions01:05 Generative AI and Task Completion02:13 Middleware and Orchestration Systems03:17 Enterprise Adoption and Challenges05:55 Multimodal AI and Future Plans08:37 AI Reliability and Evaluation11:08 Complex AI Systems and Developer Challenges13:45 Galileo's Vision and Product Roadmap18:59 Modern AI Evaluation Agents20:10 Galileo's Powerful SDK and Tools21:24 The Importance of Observability and Robust Testing22:27 The Rise of Vibe Coding24:48 Balancing Creativity and Reliability in AI31:26 Enterprise Adoption of AI Systems36:59 Challenges and Opportunities in Regulated Industries42:10 Future of AI Reliability and Industry ImpactFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Website: galileo.aiRead: Galileo Optimizes Enterprise–Scale Agentic AI Stack with NVIDIACheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

Amplitude's AI Playbook: How Wade Chambers Builds for the Agentic Future

Wed, 21 May 2025 10:00:00 GMT

As AI redefines how products are built and customers are understood, what are the core strategies engineering leaders use to drive innovation and create lasting value?Join Conor Bronsdon as he welcomes Wade Chambers, Chief Engineering Officer at Amplitude, to explore these critical questions. Wade shares how Amplitude is leveraging AI to deepen customer understanding and enhance product experiences, transforming raw data into actionable insights across their platform. He also discusses their approach to navigating constant change while building an adaptable, high-performing engineering culture that thrives in the current AI landscape.The conversation explores Amplitude's strategy for building a sustainable AI advantage through proprietary data, deep domain expertise, and robust feedback loops, moving beyond superficial AI applications. Wade offers insights on fostering an AI-ready engineering culture through empowerment and clear alignment, alongside exploring the exciting potential of agentic AI to create proactive, intelligent copilots for product teams. He then details Amplitude’s successful approach to integrating specialized AI talent, drawing key lessons from their acquisition of Command AI.Chapters00:00 Introduction and Guest Welcome01:55 Understanding and Acting on Data with AI06:42 Amplitude's Unique Position in the Market08:36 Differentiation and Competitive Advantage09:58 Incorporating Customer Feedback12:48 Evaluating AI Outcomes17:21 Agentic AI and Future Prospects21:38 Acquiring and Integrating AI Talent28:44 Building a Culture of Innovation37:21 Advice for Leaders and Individual Contributors43:26 The Future of AI in the Workplace45:38 Closing ThoughtsFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)LinkedIn: Wade ChambersWebsite: amplitude.comCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠⁠⁠Agent Leaderboard

First Code, Then AGI: Software’s Event Horizon with Poolside Founders Jason Warner & Eiso Kant

Wed, 14 May 2025 10:00:00 GMT

Is the prevailing approach to Artificial General Intelligence (AGI) missing a crucial step – deep, focused specialization? For the first time since co-founding Poolside, CEO Jason Warner & CTO Eiso Kant reunite on a podcast articulating their distinct vision for AI's future with our host, Conor Bronsdon. Poolside has intentionally diverged from general-purpose models, developing highly specialized AI meticulously designed for the specific, complex task of coding, viewing it as a direct and robust pathway towards achieving AGI, and revolutionizing how software is created.Jason and Eiso dive deep into the core tenets of their strategy: an unwavering conviction in reinforcement learning through code execution feedback and the burgeoning power of synthetic data, which they believe will help expand the surface area of software by an astounding 1000x. They candidly discuss the "devil's trade" of data privacy, Poolside's commitment to enterprise-grade AI for high-consequence systems, and why true innovation requires moving beyond flashy demos to solve real-world, critical challenges. Looking towards the horizon, they also share their insights on the evolving role of software engineers, where human agency, taste, and judgment become paramount in a landscape augmented by AI "coworkers." They also explore the profound societal implications of their work and the AI industry more generally, touching upon the "event horizon" of intelligent systems and the immense responsibility that comes with being at the forefront of this technological wave. Chapters00:00 Introduction and Guest Welcome01:19 Founding of Poolside02:56 Vision for AGI and Reinforcement Learning05:36 Defining AGI and Its Implications10:03 Training Models for Software Development17:08 Scaling and Synthetic Data20:12 Focus on High-Consequence Systems26:17 Privacy and Security in AI Solutions28:09 Earning Trust with Developers31:08 Reinforcement Learning and Compute34:29 The Vision for AI's Future39:50 Will Developers Still Exist?47:07 Poolside Cloud's Ambitions49:37 ConclusionFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Website: poolside.aiLinkedIn: Jason WarnerLinkedIn: Eiso KantCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠⁠⁠Agent Leaderboard

AI's Two Extremes – Foundations & The Frontier | Databricks’ Denny Lee

Wed, 07 May 2025 10:00:00 GMT

The AI landscape often pulls us between the allure of cutting-edge models and the quiet necessity of foundational work—yet how do these extremes actually connect to deliver value?Join Conor Bronsdon as he welcomes Denny Lee, a self-proclaimed "data nerd" and Product Management Director, Developer Relations at Dataricks, to unpack this very spectrum, from AI's core infrastructure to its most advanced applications. Denny explains why robust logging, tracing, and data lineage are indispensable for credible AI evaluation and feedback, ultimately making AI systems more affordable, accessible, and impactful.The discussion ventures into strategies for democratizing AI, exploring the "GenAI ladder" from efficient inference and retrieval-augmented generation to deciding when to fine-tune or pre-train models. Denny also tackles the industry's pressing hardware bottlenecks, the critical role of open standards, and the imperative of navigating data privacy in an increasingly AI-driven world. Listen for grounded advice on moving beyond the hype and making practical, value-driven decisions in your AI journey.Chapters00:00 Introduction and Guest Welcome01:31 Diving into AI Foundations02:25 Importance of Logging and Tracing08:40 Challenges in Data Quality and Lineage14:49 Strategies for Cost-Effective AI19:52 Partnerships and Collaborative Opportunities22:10 Hardware Bottlenecks in AI24:56 China's Power and Networking Advantage25:26 Nvidia's Super Chip and Network Fabrics26:39 The Growing Demand for Power in AI29:26 Practical Advice for Data Governance35:47 Understanding Privacy in AI36:25 Differential Privacy and Its Challenges41:57 ConclusionFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Website: Databricks.comPodcast: Data Brew by Databricks (available on major podcast platforms)YouTube: @DatabricksLinkedIn: Denny LeeReadSemiAnalysis Blog: https://semianalysis.com/Check out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠⁠⁠Agent Leaderboard

Why Enterprises Need a Different Approach to AI Agents | Lyzr’s Siva Surendira

Wed, 30 Apr 2025 10:00:00 GMT

Agentic AI exploded in 2025, but how do businesses move beyond prototypes to deploy reliable, valuable agents at scale?Join host Conor Bronsdon and Lyzr AI CEO Siva Surendira as they discuss the complexities of building and managing AI agents for enterprises. Siva shares his journey creating Lyzr, focusing on making powerful agent frameworks accessible and trustworthy for enterprise developers. They discuss the critical hurdles businesses face, including productionization challenges, ensuring responsible AI, and bridging the gap between rapid innovation and the stringent requirements of regulated industries.Listen as Siva explains Lyzr's approach to embedding safety guardrails natively and learn about the nuances of multi-agent orchestration, including managerial, DAG, and hybrid flows. Siva also offers insights into the limitations of "vibe coding" for enterprise use cases and stresses the crucial role of robust evaluation (evals) and choosing the right models—from local open-source options to frontier LLMs. Explore the bottlenecks hindering adoption, like custom application integration and data readiness, and learn why Siva believes the biggest opportunity for agent companies may not lie in replacing SaaS platforms but rather in automating the mundane work currently performed by humans.Chapters00:22 Introduction and Guest Welcome00:52 Enterprise Agent Framework02:48 Building Enterprise-Friendly AI Frameworks04:56 Enterprise Concerns with Vibe Coding09:23 Safe and Responsible AI Implementation11:05 Multi-Agent Orchestration14:13 Challenges in Multi-Agent Systems14:22 Enterprise Integration Bottlenecks17:37 The Role of Low-Code and No-Code Solutions19:55 Inter-Agent Communication Standards21:49 Future of AI Agents in Enterprises29:37 Evaluating AI Agents36:34 Conclusion and Final ThoughtsFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Website: lyzr.aiLinkedIn: Siva SurendiraCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠Agent Leaderboard

Breaking the Language Barrier: Smartling's AI Translation Pipeline | Olga Beregovaya

Wed, 23 Apr 2025 10:00:00 GMT

Are we on the verge of removing all language barriers with AI?Olga Beregovaya, VP of AI at Smartling, joins host Conor Bronsdon to tackle this question, discussing the evolution from rule-based NLP to today's powerful LLMs. Together, they confront the persistent challenges that stand in the way, like the English-centric nature of AI, domain-specific inaccuracies, and the unpredictability of model hallucinations. Olga unpacks the difficulties faced when striving for accurate, nuanced translation across all languages, especially under-resourced ones.Beyond these hurdles, the conversation explores the cutting-edge opportunities and technical innovations driving progress, including RAG, the rise of purpose-built models, agentic AI workflows, and the potential of multilingual multimodality. Olga shares insights into boosting translator productivity, achieving more predictable quality, and the path toward human parity in translation, examining how technology and human expertise will shape the future of global communication.Chapters00:00 Introduction and Guest Welcome01:14 Evolution of NLP: From Rule-Based to Machine Learning02:40 Challenges in AI Translation04:21 Biases in Language Models05:28 Inference Time and Latency05:44 English-Centric AI Models08:53 Opportunities in AI Translation09:14 Industries Benefiting from Language AI10:36 Human-in-the-Loop Translation12:06 Architectural Innovations in Language AI16:20 Success with RAG Architectures17:58 Multilingual Vectorization19:54 Agentic AI in Translation24:35 Data Sets and Data Privacy28:30 Using Smaller, Purpose-Built Models32:10 Future of AI in Translation36:37 Conclusion and FarewellFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)LinkedIn Olga BeregovayaLinkedIn ⁠SmartlingCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Low-Code AI: From Requirements to Apps in Minutes | OutSystems' Rodrigo Coutinho

Wed, 16 Apr 2025 10:00:00 GMT

What if you could turn a requirement document into a full enterprise application in just minutes?Rodrigo Coutinho, co-founder and AI Product Manager at OutSystems, joins hosts Conor Bronsdon and Atin Sanyal to explore this new reality of AI-driven development. Rodrigo shares insights from OutSystems' nearly 25-year journey, detailing their early adoption of AI and the development of their AI platform, Mentor. Discover how the pairing of AI and low-code empowers developers, accelerates the creation of enterprise applications, and shortens the cycle from idea to deployment.But this newfound speed brings its own set of challenges. The discussion addresses the hurdles of managing AI-generated code, contrasting experiences with traditional versus low-code approaches. Learn why a dev's focus pivots from syntax to strategy, pinpointing human creativity and ideation as the crucial limiter in today's development lifecycle. Chapters00:00 Welcoming Rodrigo Coutinho of OutSystems01:30 OutSystems' Early AI Journey (Pre-LLM)03:30 The LLM Revolution & OutSystems Mentor Emerges07:30 The Critical Need for Validating AI-Generated Apps12:00 The Shifting Role of the Modern Developer13:30 Quality Control & Accountability in the AI Era16:00 Low-Code's Edge in AI Validation18:30 OutSystems Mentor: A Deeper Look23:30 Choosing the Right AI Models (In-House vs Public)27:30 Future Opportunities: Speed, Experimentation & Multimodal AI37:00 The Use Case Hurdle & Final ThoughtsFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Website www.outsystems.comOutSystems MentorLinkedIn Rodrigo Sousa CoutinhoCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

AI Won't Solve Your Toughest Engineering Problems | Honeycomb’s Charity Majors

Wed, 09 Apr 2025 10:00:00 GMT

Generative AI dominates the conversation, but does it actually make it easier to build, lead, and sustain high-performing engineering teams?Host Conor Bronsdon sits down with Charity Majors, co-founder and CTO of Honeycomb (.io), and the mind behind charity.wtf. Known for her sharp insights and unfiltered opinions, Charity kicks off the discussion by expanding on her popular article: 'Generative AI is not going to build your engineering team for you.' Together, they explore how AI has altered the dynamics for engineering teams and leaders. The discussion navigates the complex dynamics of hiring in an AI-enabled era, challenging the "senior-only" trend and championing the vital role of junior engineers in creating learning organizations. Charity also explains why writing code is often the "easy part" compared to the full lifecycle of owning and operating systems, a challenge amplified by AI-generated code. Finally, Conor and Charity discuss the risk of "cognitive decay" from over-reliance on AI tools and why fostering deep system understanding remains paramount for engineers and leaders.Chapters00:00 Introduction and Guest Welcome01:51 Generative AI and Engineering Teams02:26 The Writing Process and Inspiration03:49 AI's Impact on Hiring and Team Building05:30 Embracing AI and Automation07:43 The Role of Junior Engineers09:33 Building Effective Engineering Teams17:01 Future of AI in Code Generation20:07 High Performing Engineering Teams21:48 Evolving Expectations for Engineering Managers22:41 Cognitive Decay25:00 Feedback Loops in Software Systems26:56 Hiring for Potential vs. Experience29:17 The Future of Observability39:50 Closing Thoughts and Advice for EngineersFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Follow Charity: charity.wtfLearn more about Honeycomb: www.honeycomb.ioRead: Generative AI is not going to build your engineering team for youCheck out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Inside IBM's watsonx: Building Enterprise AI That Ships | Dr. Maryam Ashoori

Wed, 02 Apr 2025 10:00:00 GMT

Building trustworthy, scalable AI isn't just about models; it's about navigating a complex ecosystem of tools and regulations. Join hosts Conor Bronsdon and Atindriyo Sanyal as they explore these challenges with Dr. Maryam Ashoori, Head of Product for watsonx AI at IBM. To meet these challenges, Maryam explains how watsonx simplifies the AI stack, automates pipelines, and empowers enterprises to scale their AI operations while optimizing costs rapidly.Maryam also explores IBM's strategy for leveraging open-source and commercial models, enabling the potential of agentic systems. Plus, she shares insights from a recent survey of 1,000 developers, revealing key takeaways about the current landscape for enterprise AI implementation, and what results mean for both developers and the enterprises they support.Chapters00:00 Introducing Dr. Maryam Ashoori01:13 Overview of IBM's AI Strategy01:47 Enterprise AI Challenges and Solutions04:40 IBM's Approach to AI Models and Tooling09:52 Simplifying the AI Stack12:20 Challenges in Agentic AI15:55 Importance of Data Management and Lineage21:11 IBM's Strategy for Gen AI Products23:43 Scaling Challenges with Agents27:40 Effective Agent Evaluation Systems35:18 Gaps and Opportunities in AI Tooling41:35 Success Stories with watsonx44:00 Closing RemarksFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)watsonx.ai Check out Galileo⁠⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Information Symmetry: DevRev's Bet on AI-Driven Enterprise Decisions | Manoj Agarwal

Wed, 26 Mar 2025 10:00:00 GMT

What if everyone in your organization had equal information at all times? Would meetings even exist? This week, we dive into the concept of information symmetry with Manoj Agarwal, co-founder and president of DevRev. Manoj, along with hosts Conor Bronsdon and Yash Sheth, explores how DevRev is connecting data, personalizing schemas, and automating complex tasks, offering a glimpse into the next generation of AI-driven workflows. This is revolutionizing enterprise data and decision-making by breaking down the silos that create information asymmetry.Learn how AI is reshaping business outcomes and collaboration, moving us closer to a world where everyone has the information they need.Chapters:00:00 Welcome to Chain of Thought00:57 Information Symmetry in Enterprises02:03 Challenges of Decision Making03:41 Recency Bias and Product Management04:58 Data Silos and Information Waste05:23 Structured vs. Unstructured Data06:04 Collaboration and Data Retrieval Issues08:17 DevRev's Approach to AI and Data Integration09:23 Building a Business-Centric Knowledge Graph10:00 Conversational AI and Automation12:57 Agentic Interactions and Skills Programming20:05 Multi-Agent Systems and Future Vision21:25 Challenges in Multi-Agent Communication25:10 Data Cleanliness and Governance28:14 Trust and Reliability in AI Systems36:58 Conclusion and Future OutlookFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)devrev.aiDevRev UniversityLinkedIn Manoj AgarwalCheck out Galileo⁠⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

The Agent Bubble Debate | Spot AI's Kelly Vaughn

Wed, 19 Mar 2025 10:00:00 GMT

Is the agentic AI bubble about to burst? Kelly Vaughn, Director of Engineering at Spot AI, questions whether the agent craze is overpromising potential and leading startups down a path of unsustainable expectations.Never one to shy away from a hot take, Kelly joins host Conor Bronsdon for a pragmatic look at AI, discussing the differences between building AI-enabled and traditional software, why replacing humans with AI teams will backfire (looking at you customer service), and the proliferation of AI tools.Kelly also shares insights on constructing AI teams, navigating data governance, and building user trust while avoiding common startup pitfalls. Chapters:00:00 Introduction and Guest Welcome01:13 Is Agentic AI a Bubble?02:40 Startup Challenges and Market Noise11:02 Building AI Products vs. Traditional Software17:31 Ethical Implications and Governance19:48 Constructing AI-Enabled Teams22:07 AI Tooling and Productivity26:00 Questioning Productivity Claims32:28 Conclusion and Final ThoughtsFollow the hostsFollow⁠⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠ Vikram⁠⁠⁠⁠⁠Follow⁠⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)Kelly’s Newsletter The Modern LeaderLinkedIn Kelly VaughnCheck out Galileo⁠⁠⁠⁠⁠⁠Try Galileo⁠⁠

Using AI to Modernize Your Legacy Applications | MongoDB’s Rachelle Palmer

Wed, 12 Mar 2025 10:35:00 GMT

Imagine cutting your legacy code modernization timeline from years to months. It’s no longer science fiction and this week’s guest is here to tell us how. Rachelle Palmer, Director of Product Management at MongoDB, joins hosts Conor Bronsdon and Atindriyo Sanyal, for a discussion on the groundbreaking ways AI is modernizing legacy applications. At MongoDB, Rachelle's forward-deployed AI engineering team is tackling the challenge of transforming complex, outdated codebases, freeing developers from technical debt. She details how LLMs are automating tasks like improving documentation, test generation, and even business logic conversion, dramatically reducing modernization timelines from years to months. What once demanded teams of dozens can now be achieved with a small, highly efficient team.Chapters:00:00 Introduction and Host Welcome00:58 Challenges in Modernizing Legacy Applications02:52 Real-World Examples of Code Modernization04:00 The Role of LLMs in Code Modernization08:01 Measuring Success in AI-Powered Modernization12:28 The Future of AI in Engineering16:17 Evaluating Modernization Success21:12 Returning to Your Startup Roots29:07 Forward Deployed AI Engineers35:36 Importance of Academic Research in AI42:10 Conclusion and FarewellFollow the hostsFollow⁠⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠ Vikram⁠⁠⁠⁠Follow⁠⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠⁠Follow Today's Guest(s)⁠Rachelle PalmerMongoDBApplication Modernization FactoryCheck out Galileo⁠⁠⁠⁠⁠Try Galileo⁠⁠

AI in 2025: Agents & The Rise of Evaluation-Driven Development

Wed, 05 Mar 2025 11:30:00 GMT

This week, we're sharing a special episode courtesy of 'Dev Interrupted.' Our co-host, Galileo CEO Vikram Chatterji, recently joined theDev Interrupted team for an engaging discussion on AI strategy. We were so impressed by the conversation that we wanted to share it with our audience, and they were kind enough to let us. We hope you enjoy it!From Dev Interrupted:"Vikram Chatterji joins Dev Interrupted’s Andrew Zigler to discuss how engineering leaders can future-proof their AI strategy and navigate an emerging dilemma: the pressure to adopt AI to stay competitive, while justifying AI spending and avoiding risky investments.To accomplish this, Vikram emphasizes the importance of establishing clear evaluation frameworks, prioritizing AI use cases based on business needs and understanding your company's unique cultural context when deploying AI."Chapters:00:00 Introduction and Special Announcement01:14 Welcome to Dev Interrupted01:42 Challenges in AI Adoption03:16 Balancing Business Needs and AI06:15 Crawl, Walk, Run Approach10:52 Building Trust and Prototyping13:07 AI Agents as Smart Routers13:50 Galileo's Role in AI Development16:25 Evaluating AI Systems25:36 Skills for Engineering Leaders27:35 Conclusion Follow the hostsFollow⁠⁠⁠⁠⁠ Atin⁠⁠⁠⁠⁠Follow⁠⁠⁠⁠⁠ Conor⁠⁠⁠⁠Follow⁠⁠⁠ Vikram⁠⁠⁠Follow⁠ ⁠⁠⁠⁠Yash⁠⁠⁠⁠⁠Follow Dev InterruptedPodcastSubstackLinkedInFollow Dev Interrupted HostsAndrewBenCheck out Galileo⁠⁠⁠⁠Try Galileo⁠⁠

The Making of Gemini 2.0: DeepMind's Approach to AI Development and Deployment | Logan Kilpatrick

Wed, 12 Feb 2025 11:00:00 GMT

Google’s strength in AI has often seemed to get lost in the midst of OpenAI announcements or DeepSeek fervor - yet Gemini 2.0 is more than good for many tasks; it’s the model to beat - and we have the research to back it up. This week, Logan Kilpatrick, senior product manager at Google DeepMind, joins us to discuss Gemini’s creation story, its emergence as the premiere model in the AI race, and why the launch of Gemini 2.0 is great news for developers.During the conversation Conor and Logan explore the exciting world of multimodal AI, Gemini's strengths in agentic use cases, and its unique approach to function calling, compositional function calling, and the seamless integration of tools like search and code execution.They also chat about Logan’s vision for a future where AI interacts with the world more naturally, offering a view of the potential of vision-first AI agents, and why Google's hardware advantage is enabling Gemini's impressive performance and long context capabilities. Follow along with the discussion using Galileo’s AI Agent Leaderboard:https://huggingface.co/spaces/galileo-ai/agent-leaderboardChapters:00:00 DeepMind's Role in Gemini's Development03:49 Gemini 2.0 Updates and Developer Highlights06:08 Agentic Use Cases and Function Calling11:29 Multimodal Capabilities16:15 Putting AI in Production21:06 Gemini's Differentiation and Hardware31:22 Future Vision for Gemini and G Suite Integration35:23 Gemini for Developers39:02 Conclusion and FarewellFollow the hostsFollow⁠⁠⁠Atin⁠⁠⁠Follow⁠⁠⁠Conor⁠⁠Follow⁠Vikram⁠Follow⁠⁠⁠Yash⁠⁠⁠Follow LoganTwitter:@OfficialLoganKLinkedIn:https://www.linkedin.com/in/logankilpatrick/Show NotesTry Gemini for yourself:gemini.google.comGemini for Developers:aistudio.google.comCheck out Galileo⁠⁠Try Galileo⁠⁠

How DeepSeek Changed the AI Race Overnight

Wed, 05 Feb 2025 11:00:00 GMT

This week, hosts Conor Bronsdon and Atindriyo Sanyal discuss the fallout from DeepSeek's groundbreaking R1 model, its impact on the open-source AI landscape, and how its release will impact model development moving forward. They also discuss what effect (if any) export controls have had on AI innovation and whether we’re witnessing the rise of “Agents as a Service”. To tackle the increasing complexity of agentic systems, Conor and Atin highlight the need for robust evaluation frameworks, discussing the challenges of measuring agent performance, and how the recent launch of Galileo's agentic evaluations are empowering developers to build safer and more effective AI agents.Chapters:00:00 Introduction02:09 DeepSeek's Impact and Innovations03:43 Open Source AI and Industry Implications13:44 Export Controls and Global AI Competition18:55 Software as a Service19:29 Agentic Evaluations 25:14 Metrics for Success31:34 Conclusion and FarewellFollow the hostsFollow ⁠⁠Atin⁠⁠Follow ⁠⁠Conor⁠Follow VikramFollow ⁠⁠Yash⁠⁠Check out GalileoTry GalileoShow NotesOn DeepSeek and Export ControlsIntroducing Agentic Evaluations

AI, Open Source & Developer Safety | Block’s Rizel Scarlett

Wed, 29 Jan 2025 11:00:00 GMT

As DeepSeek so aptly demonstrated, AI doesn’t need to be closed source to be successful.This week, Rizel Scarlett, a Staff Developer Advocate at Block, joins Conor Bronsdon to discuss the intersections between AI, open source, and developer advocacy. Rizel shares her journey into the world of AI, her passion for empowering developers, and her work on Block's new AI initiative, Goose, an on-machine developer agent designed to automate engineering tasks and enhance productivity.Conor and Rizel also explore how AI can enable psychological safety, especially for junior developers. Building on this theme of community, they also dive into topics such as responsible AI development, ethical considerations in AI, and the impact of community involvement when building open source developer tools.Chapters:00:00 Rizel's Role at Block02:41 Introducing Goose: Block's AI Agent06:30 Psychological Safety and AI for Developers11:24 AI Tools and Team Dynamics17:28 Open Source AI and Community Involvement25:29 Future of AI in Developer Communities27:47 Responsible and Ethical Use of AI31:34 Conclusion Follow Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/Rizel Scarlett : https://www.linkedin.com/in/rizel-bobb-semple/Rizel's website: https://blackgirlbytes.dev/Show NotesLearn more about Goose: https://block.github.io/goose/

AI in 2025: Agents & The Rise of Evaluation Driven Development

Wed, 15 Jan 2025 11:00:00 GMT

"In the next three to five years, every piece of software that is built on this planet will have some sort of AI baked into it." - Atin SanyalChain of Thought is back for its second season, and this episode dives headfirst into the possibilities AI holds for 2025 and beyond. Join Conor Bronson as he chats with Galileo co-founders Yash Sheth (COO) and Atindriyo Sanyal (CTO) about major trends to look for this year. These include AI finding its product "tool stack" fit, generation latency decreasing, AI agents, their potential to revolutionize code generation and other industries, and the crucial role of robust evaluation tools in ensuring the responsible and effective deployment of these agents.Yash and Atin also highlight Galileo's focus on building trust and security in AI applications through scalable evaluation intelligence. They emphasize the importance of quantifying application behavior, enforcing metrics in production, and adapting to the evolving needs of AI development.Finally, they discuss Galileo's vision for the future and their active pursuit of partnerships in 2025 to contribute to a more reliable and trustworthy AI ecosystem.Chapters:00:00 AI Trends and Predictions for 202502:55 Advancements in LLMs and Code Generation05:16 Challenges and Opportunities in AI Development10:40 Evaluating AI Agents and Applications16:07 Building Evaluation Intelligence23:41 Research Opportunities29:50 Advice for Leveraging AI in 202532:00 Closing RemarksShow Notes: Check out Galileo⁠⁠⁠⁠⁠⁠⁠⁠⁠ Follow Yash Follow Atin Follow Conor

AI Infrastructure & the Evolution of RAG | Weaviate's Bob van Luijt

Wed, 08 Jan 2025 11:00:00 GMT

"This is the time. This is the time to start building... I can't say that often enough. This is the time." - Bob van Luijt Join Bob van Luijt, CEO and co-founder of Weaviate as he sits down with our host Conor Bronson for the Season 2 premiere of Chain of Thought. Together, they explore the ever-evolving world of AI infrastructure and the evolution of Retrieval-Augmented Generation (RAG) architecture.Bob's journey with Weaviate offers a compelling example of how to adapt to rapid changes in the AI landscape. He discusses the importance of understanding developer needs and building AI-native solutions, emphasizing the potential of generative feedback loops and agent architectures to revolutionize data management.Chapters:00:00 Welcome to Season 21:43 The Evolution of AI Infrastructure04:13 Navigating Rapid Changes in AI07:39 Generative Feedback Loops and AI Native Databases13:26 Challenges and Opportunities in AI Production19:03 The Importance of Documentation and Developer Experience27:13 Future Predictions and Paradigm Shifts in AI31:17 Final Thoughts and Encouragement to BuildFollow:Conor Bronsdon: ⁠https://www.linkedin.com/in/conorbronsdon/⁠Bob van Luijt: ⁠https://www.linkedin.com/in/bobvanluijt/Weaviate: https://www.linkedin.com/company/weaviate-io/Show notes:Learn more about Weaviate: https://weaviate.io/

Beyond Chatbots: How Twilio Uses AI to Strengthen Human Connection | Vinnie Giarrusso

Wed, 18 Dec 2024 11:00:00 GMT

Can AI assistants actually enhance human connection? As Season 1 of Chain of Thought comes to a close, Conor Bronsdon and Vinnie Giarrusso (Twilio) explore the transformative potential of AI assistants in the workplace. Discover how these assistants function as "async junior digital employees," taking on specific tasks and contributing to the organizational structure. But will AI assistants ultimately replace human connection? Vinnie argues the opposite is true, suggesting that AI can liberate employees from mundane tasks, allowing them to focus on building meaningful relationships and providing personalized experiences. This thought-provoking conversation takes a philosophical turn as Vinnie explores how AI could revolutionize education while potentially disrupting traditional mentorship roles. He shares his vision for a future where AI democratizes information and empowers individuals to personalize their learning journey. Finally, learn how Twilio and Galileo are partnering to shape the future of AI and what this collaboration means for both companies. Chain of Thought will be taking a break for the holidays, but we'll see you back here on January 8th for the start of Season 2! Chapters: 00:00 Twilio's AI Agent Platform 06:34 Ensuring Accuracy and Trustworthiness 09:49 Challenges and Failure Modes 17:39 Future of Fully Autonomous Agents 22:18 Human-AI Collaboration and Mentorship 31:24 Education and Democratization of Information 32:58 Partnership with Galileo 39:54 Conclusion and Season Wrap-Up Follow: Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/ Vinnie Giarrusso: https://www.linkedin.com/in/vinniegiarrusso/ Show notes: Twilio Alpha: ⁠https://twilioalpha.com OWASP GenAI: https://genai.owasp.org

The Enterprise AI Deployment Playbook | ServiceTitan, Indeed & Twilio

Wed, 11 Dec 2024 12:00:00 GMT

This week, a panel of experts (Mehmet Murat Ezbiderli, ServiceTitan; Grant Ledford, Indeed; and Vinnie Giarrusso, Twilio) join Atin Sanyal (CTO, Galileo) and Conor Bronsdon (Developer Awareness, Galileo) to explore the challenges and opportunities of deploying GenAI at enterprise scale in a conversation that's a wake-up call for any business leader looking to harness the power of AI. Together, Atin & Conor break down key considerations like performance, cost, and model selection, emphasizing the need for robust evaluation frameworks and a shift in developer mindset. Atin then sits down with our panel of AI engineering experts to discuss their firsthand experiences with enterprise AI, including the trade-offs of building AI systems, the evolving tools and frameworks available, and the impact these technologies are having on their organizations. Chapters: 00:00 Enterprise Scale Deployment 05:17 Cost, Performance, and Model Selection 08:59 Building and Integrating GenAI Systems 15:26 Emerging Enterprise Use Cases 18:12 Predictions for AI in 2025 27:28 Panel Discussion: Deploying AI at Enterprise Scale 31:19 Gen AI Solutions and Challenges 33:12 Building & Deploying Traditional Infrastructure vs GenAI Infrastructure 34:36 How to Assemble Your GenAI Stack 40:39 Today's Best GenAI Use Cases 48:15 Enterprise AI Trends for 2025 50:36 Closing Remarks and Future Outlook Follow: Atin Sanyal: ⁠⁠⁠https://www.linkedin.com/in/atinsanyal/⁠ Mehmet Murat Ezbiderli: https://www.linkedin.com/in/mehmet-murat-ezbiderli-b894a49/ Grant Ledford: https://www.linkedin.com/in/grant-ledford-36b146a5/ Vinnie Giarrusso: https://www.linkedin.com/in/vinniegiarrusso/ Show notes: Watch all of Productionize: https://www.galileo.ai/genai-productionize-2-0

Practical Lessons for GenAI Evals | Chip Huyen & Vivienne Zhang

Wed, 04 Dec 2024 11:15:00 GMT

As AI agents and multimodal models become more prevalent, understanding how to evaluate GenAI is no longer optional – it's essential. Generative AI introduces new complexities in assessment compared to traditional software, and this week on Chain of Thought we’re joined by Chip Huyen (Storyteller, Tép Studio), Vivienne Zhang (Senior Product Manager, Generative AI Software, Nvidia) for a discussion on AI evaluation best practices. Before we hear from our guests, Vikram Chatterji (CEO, Galileo) and Conor Bronsdon (Developer Awareness, Galileo) give their takes on the complexities of AI evals and how to overcome them through the use of objective criteria in evaluating open-ended tasks, the role of hallucinations in AI models, and the importance of human-in-the-loop systems.Afterwards, Chip and Vivienne sit down with Atin Sanyal (Co-Founder & CTO, Galileo) to explore common evaluation approaches, best practices for building frameworks, and implementation lessons. They also discuss the nuances of evaluating AI coding assistants and agentic systems.Chapters:00:00 Challenges in Evaluating Generative AI05:45 Evaluating AI Agents13:08 Are Hallucinations Bad?17:12 Human in the Loop Systems20:49 Panel discussion begins22:57 Challenges in Evaluating Intelligent Systems24:37 User Feedback and Iterative Improvement26:47 Post-Deployment Evaluations and Common Mistakes28:52 Hallucinations in AI: Definitions and Challenges34:17 Evaluating AI Coding Assistants38:15 Agentic Systems: Use Cases and Evaluations43:00 Trends in AI Models and Hardware45:42 Future of AI in Enterprises47:16 Conclusion and Final ThoughtsFollow:Vikram Chatterji: https://www.linkedin.com/in/vikram-chatterji/Atin Sanyal: ⁠⁠https://www.linkedin.com/in/atinsanyal/Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/Chip Huyen: ⁠https://www.linkedin.com/in/chiphuyen/⁠Vivienne Zhang: ⁠⁠https://www.linkedin.com/in/viviennejiaozhang/Show notes:Watch all of Productionize 2.0: ⁠https://www.galileo.ai/genai-productionize-2-0⁠

Why Most Enterprise AI Projects Fail to Show ROI | HP, ServiceNow & Accenture

Wed, 27 Nov 2024 11:00:00 GMT

The “ROI of AI” has been marketed as a panacea, a near-magical solution to all business problems. Following that promise, many companies have invested heavily in AI over the past year and are now asking themselves, “What is the return on my AI investment?” This week on Chain of Thought, Galileo’s CEO, Vikram Chatterji joins Conor Bronsdon to discuss AI's value proposition, from the initial hype to the current search for tangible returns, offering insights into how businesses can identify the right AI use cases to maximize their investment. Next, we’re joined by a panel of AI experts to discuss the ROI of Enterprise AI, featuring Alex Klug, Head of Product, Data Science & AI at HP; Sriram Palapudi, Sr. Dir, ML Platform Engineering at ServiceNow; and Jay Subrahmonia, Global MD for AI Research & Products at Accenture. Together, they explore effective implementation strategies, how to measure the returns of AI adoption in the enterprise, and why AI's ROI isn't always just about the bottom line. Chapters: 00:00 Current State of AI Investments 03:59 Challenges and Solutions in AI Implementation 08:30 Identifying and Prioritizing AI Use Cases 10:53 Ensuring Trust and Explainability in AI 15:29 Measuring ROI and Efficiency Gains 21:10 Panel Discussion Begins 21:54 Trust and Risk Management at HP 23:27 Accenture's Approach to Operationalizing AI 26:06 ServiceNow's Trade-offs and Prioritization 31:17 Measuring the success of AI for customers 36:29 Frameworks and Best Practices 40:57 Conclusion and Final Thoughts Follow: Vikram Chatterji: ⁠https://www.linkedin.com/in/vikram-chatterji/ Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/ Alex Klug: https://www.linkedin.com/in/alex-klug-67ba3655/ Sriram Palapudi: https://www.linkedin.com/in/sriram-palapudi-11294b1/ Jay Subrahmonia: https://www.linkedin.com/in/jayashree-subrahmonia-99963a/ Show notes: Watch all of Productionize 2.0: ⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

GenAI Predictions for 2025 | Databricks & Cohere

Wed, 20 Nov 2024 11:00:00 GMT

Will 2025 be the year open-source LLMs catch up with their closed-source rivals? Will an established set of best practices for evaluating AI emerge? This week on Chain of Thought, we break out the crystal ball and give our biggest AI predictions for 2025. Listen as Sara Hooker, VP of Research at Cohere and Head of Cohere for AI predicts a trend towards smaller, more optimized AI models; Craig Wiley, Senior Director of Product, Mosaic AI at Databricks, dives into the future of multimodal AI; and Galileo’s CEO, Vikram Chatterji, shares his predictions, including the rise of open-source LLMs. Chapters: 00:00 Introduction 02:01 Vikram's top 3 predictions 06:19 AI and nuclear energy 08:30 Giving power back to the people 13:46 Craig's predictions 20:46 The "era of toolification" 30:38 Sara's predictions 35:07 AI safety Follow: Vikram Chatterji: ⁠⁠https://www.linkedin.com/in/vikram-chatterji/⁠ Yash Sheth: https://www.linkedin.com/in/yash-sheth-/ Conor Bronsdon: ⁠⁠https://www.linkedin.com/in/conorbronsdon/⁠⁠ Sara Hooker: https://www.linkedin.com/in/sararosehooker/ Craig Wiley: https://www.linkedin.com/in/craigwiley/ Show notes: Watch all of Productionize 2.0: ⁠⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠

Got Agents? Agentic Workflows & Architecture | Weaviate, Unstructured & CrewAI

Wed, 13 Nov 2024 11:00:00 GMT

AI agents have quickly emerged as the next ‘hot thing’ in AI, but what constitutes an AI agent and do they live up to the hype? Join Brian Raymond, founder & CEO at Unstructured.io, Bob van Luijt, co-founder & CEO at Weaviate, and João Moura, founder at CrewAI as they discuss the shift to agentic workflows, dissect their architecture, and tackle real-world challenges in agent deployment. From data management tips to generative feedback loops, this episode is your essential guide to operationalizing agents effectively. Chapters: 00:00 Defining AI Agents 01:16 Components of Agentic Architecture 02:16 Challenges and Solutions in Agent Deployment 03:58 Data Management and Quality Issues 05:23 Operationalizing Agents in Production 06:56 API and Security Considerations 09:04 Multimodal Information and Agentic Workflows 12:42 Future of Agentic Workflows 20:20 Best Practices for Agentic Strategies 25:30 Generative Feedback Loops 28:29 Agentic Evaluations Follow: Yash Sheth: https://www.linkedin.com/in/yash-sheth- Bob van Luijt: https://nl.linkedin.com/in/bobvanluijt Brian Raymond: https://www.linkedin.com/in/brian-s-raymond ⁠⁠⁠⁠⁠⁠⁠João Moura: https://br.linkedin.com/in/joaomdmoura Show notes: ⁠⁠⁠Watch all of Productionize 2.0: https://www.galileo.ai/genai-productionize-2-0

The State of AI: Open-Source Models & Enterprise Trust | May Habib

Wed, 06 Nov 2024 11:00:00 GMT

From ChatGPT's search engine to Google's AI-powered code generation, artificial intelligence is transforming how we build and deploy technology. In this inaugural episode of Chain of Thought, the co-founders of Galileo explore the state of AI, from open-source models to establishing trust in enterprise applications. Plus, tune in for a segment on the impact of the Presidential election on AI regulation. The episode culminates with an interview of May Habib, CEO of Writer, who shares practical insights on implementing generative AI at scale. Chapters: 00:00 Introduction to Chain of Thought Podcast 01:27 Big News in AI: ChatGPT and Anthropic 06:34 Open Source vs Proprietary AI 12:17 The Importance of Trust in AI 20:12 Challenges in AI Development and Deployment 22:07 The Role of Human Input in AI Development 28:45 The Future of AI Regulation 34:41 Interview with May Habib co-founder & CEO at Writer 40:01 What’s Writer’s secret sauce? 43:31 Challenges in productionizing GenAI 48:08 Conclusion Follow: Vikram Chatterji: ⁠⁠⁠https://www.linkedin.com/in/vikram-chatterji/⁠⁠ Atin Sanyal: ⁠https://www.linkedin.com/in/atinsanyal/ Yash Sheth: ⁠https://www.linkedin.com/in/yash-sheth-/⁠ Conor Bronsdon: ⁠⁠⁠https://www.linkedin.com/in/conorbronsdon/⁠⁠⁠ May Habib: https://www.linkedin.com/in/may-habib/ Show notes: Watch all of Productionize 2.0: ⁠⁠⁠⁠https://www.galileo.ai/genai-productionize-2-0⁠⁠