SB 1047: California's AI Safety Bill

In 2024, California almost passed the first US state law targeting frontier AI safety. Governor Newsom vetoed it. The fight reshaped the AI policy landscape.

42 min · Reviewed 2026

What the Bill Proposed

California's Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, introduced by State Senator Scott Wiener in February 2024, would have applied to 'covered models' — models trained with more than 10^26 FLOPs or costing over $100 million to train. It required safety testing, a kill switch, and legal liability for catastrophic harms caused by models that had not taken reasonable precautions.

Key provisions

Safety and security protocols submitted to the state Attorney General
A capability to fully shut down a covered model
Third-party auditing of safety practices
Whistleblower protections for lab employees
Tort liability for serious harms where developers acted unreasonably
A Board of Frontier Models within the California Government Operations Agency

The veto and aftermath

The bill passed both California chambers in August 2024. On September 29, 2024, Governor Gavin Newsom vetoed it. His veto message argued the bill's size threshold was a poor proxy for risk and that narrower, more technical regulation was preferable. He commissioned a working group, which released recommendations in 2025.

SB 53 (Wiener, 2025): narrower follow-up focusing on transparency and whistleblowers — signed into law
Frontier labs published or expanded their Responsible Scaling Policies in 2024-2025
Federal AI Action Plan and executive orders filled some of the gap
Other states (Colorado, New York) moved on different AI regulations

By focusing only on the most expensive and large-scale models, SB 1047 establishes a regulatory framework that could give the public a false sense of security.
— Governor Gavin Newsom, veto message, September 2024

The big idea: SB 1047 did not become law, but it established the vocabulary and battle lines for every frontier AI regulation that follows. Read the bill if you want to understand current policy debates.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-safety2-sb1047-creators

Which California state legislator introduced SB 1047?
1. State Assemblymember Rebecca Bauer-Kahan
2. State Senator Scott Wiener
3. Governor Gavin Newsom
4. Attorney General Rob Bonta
Which of these was a key requirement for developers under SB 1047?
1. Sharing training data with the federal government
2. Hiring a government-appointed AI ethicist
3. Implementing a full 'kill switch' to shut down the model
4. Publishing model weights publicly
Which of the following individuals or organizations SUPPORTED SB 1047 (with amendments)?
1. Meta and Andreessen Horowitz
2. OpenAI and Google
3. Anthropic
4. Fei-Fei Li and most of California's congressional delegation
Which organization called SB 1047 a threat to 'open source' AI development?
1. The Electronic Frontier Foundation
2. OpenAI
3. The Center for AI Safety
4. Anthropic
What was Governor Newsom's main criticism of SB 1047 in his veto message?
1. The size threshold was a poor proxy for risk
2. The bill gave too much power to the Attorney General
3. The bill was unconstitutional
4. The bill was too weak on AI safety
Which follow-up bill did Senator Wiener introduce in 2025 that focused on transparency and whistleblower protections?
1. AB 2930
2. SB 1047
3. SB 891
4. SB 53
What was the 'Board of Frontier Models' that SB 1047 would have created?
1. A research ethics committee at Stanford University
2. A board within the California Government Operations Agency
3. An industry self-regulatory body
4. A federal advisory committee
Which of these was NOT mentioned as a key provision of SB 1047?
1. Mandatory public release of training data
2. Third-party auditing of safety practices
3. Whistleblower protections for lab employees
4. Tort liability for serious harms
What is a 'Responsible Scaling Policy' in the context of frontier AI labs?
1. An internal company document outlining safety commitments and thresholds
2. A type of machine learning algorithm
3. A federal safety standard
4. A legal document required by California law
Which prominent AI researcher was among the supporters of SB 1047?
1. Geoffrey Hinton
2. Yann LeCun
3. Demis Hassabis
4. Sam Altman
What was the approximate date when Governor Newsom vetoed SB 1047?
1. August 2024
2. September 2024
3. June 2024
4. December 2024
Which argument against SB 1047 claimed the bill regulated the wrong aspect of AI systems?
1. It regulated model capabilities rather than deployment
2. It regulated open-source models
3. It regulated the wrong layer
4. It regulated too late in the development process
What type of harms did SB 1047 aim to assign tort liability for?
1. Copyright violations
2. Minor inconveniences to users
3. Any competitive business losses
4. Catastrophic harms where developers acted unreasonably
After SB 1047 was vetoed, which states moved forward with different AI regulations?
1. Colorado and New York
2. Illinois and Massachusetts
3. Washington and Oregon
4. Texas and Florida
What was the 'big idea' the lesson identified as SB 1047's lasting impact?
1. It established the first federal AI safety standard
2. It created the template for all future AI regulation globally
3. It established the vocabulary and battle lines for frontier AI regulation
4. It proved that state-level AI legislation is unconstitutional

← Back to interactive lesson

Tendril · Creators · Ethics & Society