SACROMENTO, Calif. (AP) — California Governor Gavin Newsom signed a law Monday aimed at preventing people from using powerful artificial intelligence models for potentially catastrophic activities such as building Bioweapon and closing banking systems.
The move comes as Newsom promoted California as the leader in AI regulation and criticised the federal level of inaction in a recent conversation with former President Bill Clinton. The new law would establish some of the national first-condition regulations on large-scale AI models without damaging the state’s own industry, Newsom said. Many of the world’s top AI companies are in California and must follow the requirements.
“California has proven that it can establish regulations to protect our communities and ensure that the growing AI industry continues to thrive. This law balances that,” Newsom said in a statement.
The law requires AI companies to implement and disclose public safety protocols to ensure that the most sophisticated models are not used. The rules are designed to cover cases where AI systems meet “frontier” thresholds that run with enormous amounts of computing power.
Such a threshold is based on the number of calculations the computer is doing. Those who created the regulations acknowledge that numerical thresholds are an incomplete starting point for distinguishing them from the next generation, which could potentially be even stronger than today’s best-generated AI systems. Existing systems are primarily created by California-based companies such as Humanity, Google, Meta Platforms, and Openai.
The law defines catastrophic risk as causing at least $1 billion in damage or more than 50 injuries or deaths. It is designed to prevent AI used for activities that can cause a lot of confusion, such as hacking into power grids.
Businesses must also report important safety incidents to the state within 15 days. The law creates whistleblower protections for AI workers and establishes a public cloud for researchers. A $1 million fine is included for each violation.
It drew opposition from some tech companies. This argued that AI laws should be carried out at the federal level. However, humanity said that regulations are “practical safety measures” and that they officially create safety practices that many companies have already implemented voluntarily.
“While federal standards are essential to avoid a patchwork of state regulations, California has created a strong framework that balances public safety with ongoing innovation,” said Jack Clark, policy director and policy director for humanity, in a statement.
The signature comes after Newsom rejected a broader version of the law last year, refusing to sid with tech companies that it said the requirements were too strict and hampering innovation. Instead, Newsom has asked a group of several industry experts, including the AI Pioneer Fei-Fei Li, to develop guardrail recommendations around powerful AI models.
The new law incorporates recommendations and feedback from Newsom’s AI experts and industry groups, proponents said. San Francisco Senator Scott Wiener, author of the bill, said the law also does not have the same level of reporting requirements for startups to avoid damaging innovation.
“With this legislation, California is stepping up once again as a global leader in both technological innovation and safety,” Wiener said in a statement.
Newsom’s decision comes when President Donald Trump announced plans in July to eliminate what his administration considers as “snagging” regulations to speed up AI innovation and solidify America’s position as a global AI leader. Earlier this year, Congressional Republicans tried to ban states and regions from regulating AI for a decade.
Without stronger federal regulations, states across the country have been trying to hold back their technology for the past few years, working on everything from deep fakes of elections to AI “treatment.” In California, this year, Congress passed numerous bills to address safety concerns regarding AI chatbots for children and the use of AI in the workplace.
California is also an early adopter of AI Technologies. The state is deploying generation AI tools to spot wildfires and deal with highway congestion and road safety.
___
Associated Press reporter Matt O’Brien contributed to the report.
