Data Center

When you ask a chatbot a question, the answer is computed in a data center: a warehouse-sized building of servers, connected by fast networks and kept running by large power supplies and cooling systems. Companies can own one, rent space in a shared “colocation” facility, or rent capacity from a cloud hyperscaler such as AWS, Microsoft Azure or Google Cloud.

AI data centers differ from traditional ones mainly in density. A rack of AI accelerators draws far more power and gives off far more heat than a rack of ordinary web servers, so these sites often need liquid cooling and new grid connections. Training a frontier model can tie up tens of thousands of GPUs for weeks, so the networking between machines matters as much as the chips. Serving the model to users (inference) then needs capacity spread across many regions.

Why it matters: data centers are where AI’s compute costs turn into physical limits such as land, electricity, water and chips. The International Energy Agency estimates data centres used about 1.5% of the world’s electricity in 2024, or 415 TWh. Its 2025 report projects their electricity consumption to more than double to about 945 TWh by 2030, with AI the biggest driver. It also stresses that local effects on particular grids are much larger than the global share suggests. For a business, data center location also affects latency and where data is stored, which matters for data sovereignty rules.

Guests on the show come at this from different angles. In episode 44 at 34:24, Cisco’s Jeetu Patel says a single data center “can only host fifty, seventy thousand, a 100,000 GPUs,” so the largest training runs may need clusters that span several sites. In episode 45 at 3:22, writer Andy Masley argues AI data centers “don’t seem to be a significant issue for water” in the United States; that is his conclusion from his own research, and the episode opens with a Galileo sponsor read. In episode 62 at 17:04, Kris Lovejoy describes grid operator PJM repeatedly underestimating load because of new data center requests, and says the facilities are often efficient; the growth comes from sheer scale.