PyTorch handles the heavy machinery so a researcher can write a model in a few dozen lines of Python. It stores numbers in tensors, runs the math on GPUs or other accelerators, and computes the gradients that backpropagation needs to train a neural network automatically. A typical project downloads a pretrained model from Hugging Face, loads it into PyTorch and fine-tunes it on new data.
Its main selling point when it appeared was that it ran like normal Python: each line executes immediately, so code is easy to inspect and debug. That made it popular with researchers, and research code tends to become production code. Meta moved PyTorch to a foundation under the Linux Foundation in 2022 so that governance is shared across companies.
Because so much AI code is written in PyTorch, support for it is a requirement for any new chip. Under the hood, PyTorch on NVIDIA hardware relies on CUDA, and competing vendors work to make the same PyTorch code run on their chips. The switching cost in AI hardware is software explains why.
In episode 72 at 9:13, AMD’s Anush Elangovan says frameworks “like PyTorch, VLLM, SG-Line” (the transcript’s spellings of vLLM and SGLang) now have full enablement on AMD, with continuous testing that can block upstream changes that would break AMD support. That is AMD’s account of its own progress. Episode 72 names Svix and Walrus Memory as presenting sponsors in its opening read. In episode 63 at 34:05, Dhruv Batra cites PyTorch as an industry innovation “invented for internal” use and then shared freely.
Hear it from the guest
“But most of the frameworks like PyTorch, VLLM, SG-Line, all of those are now currently like 100% … like we have 100% enablement.”
“PyTorch was invented for internal”
Quotes lightly edited to remove filler words.
Sources
- Paszke et al.: PyTorch: An Imperative Style, High-Performance Deep Learning Library (NeurIPS 2019) — The original paper describing PyTorch's design: ordinary Python code that runs immediately, with automatic differentiation and GPU acceleration; checked October 24, 2026.
- PyTorch Foundation — States that the PyTorch Foundation is hosted by the Linux Foundation and supports the core PyTorch framework alongside other projects under open governance; checked October 24, 2026.
