In detail
1. The moat is day-one model support
AMD’s Anush Elangovan describes what the alternative platform used to feel like: ROCm support was a “port to platform.” Someone would launch a model, and then someone else would go in and make it work. The change he points to is that the recent frontier releases (DeepSeek, Llama, Qwen) run natively from day zero, as well as they do on the competing platform. That confidence is what a buyer is paying NVIDIA for: a model announced this morning runs on their cluster this afternoon.
Update, October 3, 2026: At 9:13 in episode 72, Anush Elangovan describes AMD’s expanded testing across frameworks and hardware generations. Elangovan says AMD can block upstream pull requests and test them on AMD hardware to help prevent regressions from changes tested only on NVIDIA. For a team evaluating the day-one support discussed above, upstream testing is an ongoing compatibility check to ask about.
2. The specs are the catchable part
Hardware differences show up on a roadmap and get closed on a roadmap. AMD’s CDNA4 generation added FP6 as a stepping stone toward FP4, so a team can move parts of a model down in precision in stages. The 350 series ships 288 GB of memory per GPU, which on AMD’s own framing puts a 500-billion-parameter model on a single card. NVIDIA’s Blackwell Ultra now matches that 288 GB, and AMD’s newer MI455X, launched in July 2026, raises it to 432 GB. Memory capacity is where AMD keeps pressing its case. None of that, by itself, moves a workload.
3. The other exit is owning the whole stack
Google took the path almost nobody else can afford. TPUs sit under XLA as the target compiler, and JAX sits on top of XLA, the framework Google DeepMind uses in research and in production. Paige Bailey’s point is that JAX is loved partly because it uses TPUs so well, and scales not just across many TPUs but across multiple data centers, which is what training Gemini requires. Vertical integration from silicon to framework is why the optimizations available to Google are not available to a team renting someone else’s stack.
4. Open is the counter-strategy, and it is a software strategy
AMD’s bet is that an open ecosystem out-innovates a closed one, and the tell is where they applied it: the software review process is moving outside the company, so any developer working in an area reviews the changes in it. Partners build switches, NICs, and interconnects around them rather than at the periphery of a proprietary fabric. The pitch is about the stack rather than the accelerator: you should not have to buy the whole thing from the company that sells you the chip. AMD VP of AI Software Anush Elangovan says he keeps the entire ROCm codebase checked out on a Strix Halo machine, so he can make one change and have an agent carry it across the whole stack.
Check what a fit claim leaves out
The 500-billion-parameter figure above depends on the number format. At four bits per weight, 500 billion weights take 500 billion × 4 ÷ 8 = 250 billion bytes, or 250 GB, which fits in 288 GB. At 16 bits the same weights take 1,000 GB. Neither figure counts quantization metadata, layers kept at higher precision, or the KV cache that grows with input length and concurrency during serving, and training also needs room for activations, gradients and optimizer state. Measure peak memory on the configuration you would actually run.
The code may need fewer changes than its spelling suggests. PyTorch’s ROCm build “intentionally reuses the existing torch.cuda interfaces”, so cuda in a device string does not by itself tie code to NVIDIA, and torch.version.hip tells the two builds apart. Custom kernels, vendor libraries and distributed communication still need checking one at a time, and AMD’s compatibility matrix notes that “ROCm depends on a coordinated stack of compatible firmware, driver, and user space components.” A short inventory before switching:
| Area | What to confirm |
|---|---|
| Platform | Accelerator, OS, driver, firmware, runtime and container versions |
| Model execution | Framework and serving-engine versions, number formats and custom extensions |
| Distributed work | Collective operations, network topology and data loading at the target scale |
| Operations | Profiling, monitoring, upgrade tests, a named owner for regressions and a rollback path |
Why it matters
If you are evaluating accelerators on FLOPS and memory, you are comparing the part that converges. Ask instead how long after a model release your team can run it, who fixes the kernel when it breaks, and what you would have to rewrite to leave. That answer is the switching cost.
Hear it from the guest
“The internal source repositories and external source repositories are exactly the same. It is all external, right? And so what that means is, as an external developer, you can contribute any code changes you want and we actually take that seriously and merge it in.”
“Why people love JAX so much other than JAX being just like a friendly NumPy like interface to do work is because it can use TPUs so optimally.”
“We can block PRs that go in and are always tested on AMD”
“If there's nothing open to look at, you're kind of, like, touching around a black box and not really learning what's going on.”
“I literally have a Strix Halo with the entire Rokken codebase checked out. And I could make one change and I could say, go make this change across the entire stack.”
Quotes lightly edited to remove filler words.
Go deeper
- ROCm compatibility matrix Check a proposed AMD deployment against supported GPU, operating-system and framework combinations. Compatibility is a prerequisite for a workload benchmark.
- Matrix Multiplication Work through a tiled GPU kernel, cache-aware ordering and automatic tuning. The example includes CUDA and HIP configurations; it does not establish portability for a whole application.
From the conversation
This explainer is drawn from these episodes — each carries its full transcript.
-
AMD's Challenge to NVIDIA: The Open Ecosystem Bet | Anush Elangovan & Sharon Zhou -
Gemini 3 & Robot Dogs: Inside Google DeepMind's AI Experiments | Paige Bailey -
Tuning GPU Performance with AI Agents | AMD’s Anush Elangovan on ROCm 10 -
250,000 Lines of Code/Week: Inside an AMD VP's Agent-First Workflow | Anush Elangovan
Concepts in this explainer
PyTorchCUDAFLOPsFloating-point precision and mixed precisionKV CacheAI InterconnectTPU (Tensor Processing Unit)Frontier Model