Most AI work boils down to multiplying large grids of numbers. A regular CPU can do that, but slowly. Accelerators do thousands of those multiplications in parallel. Graphics processing units (GPUs) are the best-known example. Google’s tensor processing units (TPUs) are chips built specifically for machine learning, and many phones and laptops now include a neural processing unit (NPU) for on-device AI features such as live transcription or photo search.
Why it matters
Accelerators decide how fast a model answers, how much each answer costs and whether a model can run on a device at all. When a company says it can sell a model cheaply, part of that story is usually its hardware.
On the podcast
In episode 12, Google’s Logan Kilpatrick connects TPU progress to price, saying “It’s because we actually have control of the hardware too” at 24:30. That is his account of Google’s advantage, not an independent cost analysis. Galileo sponsored episode 12.
In episode 70, Tormod Ree describes meeting-room devices that run a “system on chip” combining CPU, NPU and GPU, and says GPU and NPU acceleration has improved much faster than CPUs at 37:38. Svix and Walrus Memory were season-four presenting sponsors; episode 70’s show notes carry their links.
Related terms
A GPU and a TPU are specific kinds of accelerator. Inference is the work most deployed accelerators spend their time on.
Hear it from the guest
“It's because we actually have control of the hardware too”
“So typically our devices, they run sort of these system on chip.”
Quotes lightly edited to remove filler words.
Sources
- Google Cloud: Introduction to Cloud TPU — Google describes TPUs as custom-designed application-specific integrated circuits used to accelerate machine learning workloads; checked October 24, 2026.
- Google Cloud: TPU architecture — Explains how TPU matrix-multiply units perform the large matrix operations that dominate neural-network training and inference; checked October 24, 2026.
