AI Hardware
The chips and accelerators that make AI run.
AI hardware is the physical layer beneath modern AI — the GPUs, accelerators, and silicon that train and serve models, and the competition among vendors to deliver more compute per watt and per dollar.
Hardware decisions need workload evidence. AMD's ROCprofiler-SDK documentation distinguishes application tracing from collecting low-level GPU performance counters, and covers kernel dispatches and memory moves (AMD, ROCprofiler-SDK documentation). For an inference deployment, profile the model and request sizes you intend to run before choosing what to optimize. Keep correctness checks beside timing measurements so a faster implementation still meets the application's requirements.
Anush Elangovan in episode 72 describes finding model hotspots before trying kernel changes. Tormod Ree in episode 70 discusses distributing compute across meeting-room devices. Why is it hard to move off NVIDIA? covers the surrounding software ecosystem; are small language models better for production? connects model choice to deployment constraints. Autotuning defines the search-and-measure loop used to choose implementations.
Start here
- Tuning GPU Performance with AI Agents | AMD’s Anush Elangovan on ROCm 10
See how profiling finds model hotspots before GPU performance tuning.
- 11 Cameras, Dozens of Mics: The AI That Reads the Room | Tormod Ree, Neat
Hear why a meeting device ships a combined CPU, NPU, and GPU chip and spreads compute across the room.
Go deeper
Scaling GPU systems
17 episodes
- Genspark's Bet: AI Agents Become the Users of Software | Wen Sang
- Tuning GPU Performance with AI Agents | AMD’s Anush Elangovan on ROCm 10
- 11 Cameras, Dozens of Mics: The AI That Reads the Room | Tormod Ree, Neat
- Data Federation, Not Centralization, Is What Enterprise AI Needs | Jitender Aswani, Starburst
- You Can't Secure an AI Agent with Software | Charles Guillemet, Ledger
- Most of the Web Will Never Get APIs for AI Agents | Dhruv Batra, Yutori
- 250,000 Lines of Code/Week: Inside an AMD VP's Agent-First Workflow | Anush Elangovan
- Gemini 3 & Robot Dogs: Inside Google DeepMind's AI Experiments | Paige Bailey
- The Critical Infrastructure Behind the AI Boom | Cisco CPO Jeetu Patel
- Beyond Transformers: How Liquid AI Is Rethinking LLM Architecture | Maxime Labonne
- From Demo to Defensibility: How to Build an AI Business that Lasts | Aurimas Griciūnas
- AMD's Challenge to NVIDIA: The Open Ecosystem Bet | Anush Elangovan & Sharon Zhou
- Why Gamers Paved the Way for AI | Databricks' Carly Taylor
- AI's Two Extremes – Foundations & The Frontier | Databricks’ Denny Lee
- The Making of Gemini 2.0: DeepMind's Approach to AI Development and Deployment | Logan Kilpatrick
- How DeepSeek Changed the AI Race Overnight
- Practical Lessons for GenAI Evals | Chip Huyen & Vivienne Zhang
Explainers on this topic
- Why AI Runs on GPUs
- Physical AI: When a Language Model Drives the Robot
- The Switching Cost in AI Hardware Is Software
Terms on this topic
- AI Chip Export Controls
- AI Interconnect
- Application-Specific Integrated Circuit (ASIC)
- Autotuning
- Batch Size
- Compute
- CUDA
- Data Center
- Edge AI
- Field-Programmable Gate Array (FPGA)
- Floating-point precision and mixed precision
- FLOPs
- Graphics Processing Unit
- High-Bandwidth Memory (HBM)
- Moore's Law
- Neocloud
- Neuromorphic Computing
- Quantum Computing
Guests on this topic
Wen SangAnush ElangovanTormod ReeJitender AswaniCharles GuillemetDhruv BatraPaige BaileyJeetu PatelMaxime LabonneAurimas GriciūnasSharon ZhouCarly TaylorDenny LeeLogan KilpatrickAtindriyo SanyalChip HuyenVivienne Zhang