The Environmental Cost of Training a Big Model

Training a frontier model uses the electricity of a small city for months. Running inference at scale matches a large country's load. Here is what the numbers actually look like.

25 min · Reviewed 2026

Compute Has a Physical Footprint

Every token your favorite AI generates came out of a GPU that was drawing electricity from a grid that was, somewhere upstream, burning something or spinning something. The abstraction is clean. The physics is not.

Training numbers you should actually know

GPT-3 (2020): estimated ~1,287 MWh to train, roughly 500 metric tons of CO2
GPT-4 class (2023): publicly estimated at tens of thousands of MWh
Frontier 2025 training runs: hundreds of GWh, matching small-city annual consumption
Llama 3 70B: Meta disclosed ~1,900 MWh
Training electricity is a one-time cost — inference is forever

Inference is the real long-term story

Training a model takes weeks or months and then stops. Running it serves billions of users every day for years. By 2025, most analyses estimated that inference consumed more energy than training across the industry. The IEA projected that global data center electricity use could reach 945 TWh by 2030, with AI as the fastest-growing slice.

Water is the other quiet variable

Data centers are cooled by enormous amounts of water. A 2023 study estimated GPT-3 training consumed about 700,000 liters of fresh water. Microsoft's water use jumped 34 percent from 2021 to 2022, partly attributed to AI. In drought-prone regions like Arizona, this has become a political fight.

Compare: common computing footprints

Activity	Rough energy cost
Google search	~0.3 Wh
LLM chat turn	~3-10 Wh
Image generation	~30-100 Wh
Video generation (per second)	~300-1000 Wh
Train GPT-3 once	~1.3 GWh
Train frontier 2025 model	100+ GWh

What is changing

Model efficiency: smaller specialist models often beat giant ones per task
Sparse mixture of experts: only activate parts of a model per query
Custom silicon: TPUs, Trainium, Cerebras, Groq all target more ops per watt
Renewable siting: data centers are chasing hydro, geothermal, and nuclear
Nuclear renaissance: Amazon, Google, Microsoft all signed nuclear power deals in 2024-2025

What is not changing

Demand is growing faster than efficiency
New AI features create new inference load, not less
Training runs are getting bigger, not smaller
Grid buildout takes a decade; model training takes months

You cannot compute your way out of thermodynamics. Somebody always pays the electric bill.
— A data center engineer

The big idea: AI has real physical costs that are often hidden behind a clean chat interface. Whether they are worth it is a judgment call — but you cannot make the call without knowing the numbers.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-ethics-environmental-cost-builders

What is the main idea of "The Environmental Cost of Training a Big Model"?
1. Training a frontier model uses the electricity of a small city for months.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "The Environmental Cost of Training a Big Model"?
1. electricity
2. compute cost
3. water cooling
4. carbon
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. GPT-3 (2020): estimated ~1,287 MWh to train, roughly 500 metric tons of CO2
4. Use the first answer without checking it
What should a careful learner remember about "Per-query numbers"?
1. Use AI to draft or organize ideas about compute cost, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. AI cannot make the human values decision for you.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about compute cost be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about compute cost.
Which action would help you apply "The Environmental Cost of Training a Big Model" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Use the first answer without checking it
4. GPT-4 class (2023): publicly estimated at tens of thousands of MWh

← Back to interactive lesson

Tendril · Builders · Ethics & Society