The Environmental Cost of Training a Big Model

Every token your favorite AI generates came out of a GPU that was drawing electricity from a grid that was, somewhere upstream, burning something or spinning something. The abstraction is clean. The physics is not.

Training numbers you should actually know

• GPT-3 (2020): estimated ~1,287 MWh to train, roughly 500 metric tons of CO2
• GPT-4 class (2023): publicly estimated at tens of thousands of MWh
• Frontier 2025 training runs: hundreds of GWh, matching small-city annual consumption
• Llama 3 70B: Meta disclosed ~1,900 MWh
• Training electricity is a one-time cost — inference is forever

Inference is the real long-term story

Training a model takes weeks or months and then stops. Running it serves billions of users every day for years. By 2025, most analyses estimated that inference consumed more energy than training across the industry. The IEA projected that global data center electricity use could reach 945 TWh by 2030, with AI as the fastest-growing slice.

Water is the other quiet variable

Data centers are cooled by enormous amounts of water. A 2023 study estimated GPT-3 training consumed about 700,000 liters of fresh water. Microsoft's water use jumped 34 percent from 2021 to 2022, partly attributed to AI. In drought-prone regions like Arizona, this has become a political fight.

Compare: common computing footprints

What is changing

• Model efficiency: smaller specialist models often beat giant ones per task
• Sparse mixture of experts: only activate parts of a model per query
• Custom silicon: TPUs, Trainium, Cerebras, Groq all target more ops per watt
• Renewable siting: data centers are chasing hydro, geothermal, and nuclear
• Nuclear renaissance: Amazon, Google, Microsoft all signed nuclear power deals in 2024-2025

What is not changing

• Demand is growing faster than efficiency
• New AI features create new inference load, not less
• Training runs are getting bigger, not smaller
• Grid buildout takes a decade; model training takes months

“You cannot compute your way out of thermodynamics. Somebody always pays the electric bill.”

The big idea: AI has real physical costs that are often hidden behind a clean chat interface. Whether they are worth it is a judgment call — but you cannot make the call without knowing the numbers.