SB 1047: California's AI Safety Bill

In 2024, California almost passed the first US state law targeting frontier AI safety. Governor Newsom vetoed it. The fight reshaped the AI policy landscape.

42 min · Reviewed 2026

What the Bill Proposed

California's Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, introduced by State Senator Scott Wiener in February 2024, would have applied to 'covered models' — models trained with more than 10^26 FLOPs or costing over $100 million to train. It required safety testing, a kill switch, and legal liability for catastrophic harms caused by models that had not taken reasonable precautions.

Key provisions

Safety and security protocols submitted to the state Attorney General
A capability to fully shut down a covered model
Third-party auditing of safety practices
Whistleblower protections for lab employees
Tort liability for serious harms where developers acted unreasonably
A Board of Frontier Models within the California Government Operations Agency

The veto and aftermath

The bill passed both California chambers in August 2024. On September 29, 2024, Governor Gavin Newsom vetoed it. His veto message argued the bill's size threshold was a poor proxy for risk and that narrower, more technical regulation was preferable. He commissioned a working group, which released recommendations in 2025.

SB 53 (Wiener, 2025): narrower follow-up focusing on transparency and whistleblowers — signed into law
Frontier labs published or expanded their Responsible Scaling Policies in 2024-2025
Federal AI Action Plan and executive orders filled some of the gap
Other states (Colorado, New York) moved on different AI regulations

By focusing only on the most expensive and large-scale models, SB 1047 establishes a regulatory framework that could give the public a false sense of security.
— Governor Gavin Newsom, veto message, September 2024

The big idea: SB 1047 did not become law, but it established the vocabulary and battle lines for every frontier AI regulation that follows. Read the bill if you want to understand current policy debates.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-safety2-sb1047-creators

What is the main idea of "SB 1047: California's AI Safety Bill"?
1. In 2024, California almost passed the first US state law targeting frontier AI safety.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "SB 1047: California's AI Safety Bill"?
1. frontier AI
2. SB 1047
3. AI regulation
4. covered model
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Safety and security protocols submitted to the state Attorney General
4. Treat the AI output as automatically correct
What should a careful learner remember about "The debate"?
1. Use "The debate" as a reminder to verify the AI output before anyone relies on it.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. AI cannot make the human values decision for you.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about SB 1047 be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about SB 1047.
Which action would help you apply "SB 1047: California's AI Safety Bill" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. A capability to fully shut down a covered model

← Back to interactive lesson

Tendril · Creators · Ethics & Society