Application-Specific Integrated Circuit (ASIC)

A CPU can run almost any program. A GPU gives up some of that generality to do huge amounts of parallel math. An ASIC goes further: its circuits are laid out for a narrow set of operations, such as the matrix multiplications at the heart of neural networks, and little else. Done well, that means more useful work per chip and per watt for the job it was built for.

The catch is commitment. Designing a leading-edge chip takes years and a very large upfront investment, and the design cannot be changed once it is manufactured. If model architectures shift in a direction the chip did not anticipate, it can age badly. That is why ASICs make most sense for companies running enormous, predictable AI workloads. Google’s TPU, introduced in its data centers in 2015, is the best-known AI ASIC; AWS’s Trainium follows the same logic. Software support decides whether outsiders can use them; see CUDA for why that matters.

ASICs are not only for model math. Networking switches, video encoders and the chips in a phone that handle specific tasks are ASICs too. A field-programmable gate array (FPGA) sits between the two extremes: specialized hardware that can be rewired after manufacturing.

In episode 44 at 46:12, Cisco’s Jeetu Patel says Cisco “will build silicon and network ASICs” for low-latency, energy-efficient networks, as part of his pitch for Cisco as critical AI infrastructure. During the episode, Conor notes that the show’s sponsor at the time, Galileo, is an NVIDIA AI factory partner (31:57).