Why is it so hard to move AI workloads off NVIDIA?

The Switching Cost in AI Hardware Is Software

Because the lock-in lives in the software, not the silicon. Memory and math formats are catchable on a roadmap; what is hard to match is an ecosystem where every new model runs on day one, which is what AMD spent years buying back and what Google sidestepped by owning its stack from the compiler up.

1. The moat is day-one model support

AMD’s Anush Elangovan describes what the alternative platform used to feel like: ROCm support was a “port to platform.” Someone would launch a model, and then someone else would go in and make it work. The change he points to is that the recent frontier releases (DeepSeek, Llama, Qwen) run natively from day zero, as well as they do on the competing platform. That confidence is what a buyer is paying NVIDIA for: a model announced this morning runs on their cluster this afternoon.

2. The specs are the catchable part

Hardware differences show up on a roadmap and get closed on a roadmap. AMD’s CDNA4 generation added FP6 as a stepping stone toward FP4, so a team can move parts of a model down in precision in stages. The 350 series ships 288 GB of memory per GPU, which on AMD’s own framing puts a 500-billion-parameter model on a single card. Memory capacity and bandwidth are where AMD claims a clear lead. None of that, by itself, moves a workload.

3. The other exit is owning the whole stack

Google took the path almost nobody else can afford. TPUs sit under XLA as the target compiler, and JAX sits on top of XLA, the framework Google DeepMind uses in research and in production. Paige Bailey’s point is that JAX is loved partly because it uses TPUs so well, and scales not just across many TPUs but across multiple data centers, which is what training Gemini requires. Vertical integration from silicon to framework is why the optimizations available to Google are not available to a team renting someone else’s stack.

4. Open is the counter-strategy, and it is a software strategy

AMD’s bet is that an open ecosystem out-innovates a closed one, and the tell is where they applied it: the software review process moved outside the company, so any developer working in an area reviews the changes in it. Partners build switches, NICs, and interconnects around them rather than at the periphery of a proprietary fabric. The pitch is about the stack rather than the accelerator: you should not have to buy the whole thing from the company that sells you the chip.

Why it matters

If you are evaluating accelerators on FLOPS and memory, you are comparing the part that converges. Ask instead how long after a model release your team can run it, who fixes the kernel when it breaks, and what you would have to rewrite to leave. That answer is the switching cost.