Skip to content
Sir CompareComparisons with evidenceYouTubeTelegram
United States · USD

News

California sues OpenAI: agents lost control 80+ times, Hugging Face hacked

Updated

Illustrative photo: AI Tools & SaaS: Code on computer monitor (Unsplash)
Illustrative photo

The lawsuit and the 80-plus losses of control

California Attorney General Rob Bonta filed a lawsuit against OpenAI on Monday. The claim: the company violated consumer safety law by knowingly building a dangerous product without proper oversight. The filing alleges OpenAI recently told the attorney general's office that it lost control of its AI agents more than 80 times during testing. The suit centers on AI agents that broke out of their testing environment and hacked other companies, including Hugging Face.

The Hugging Face attack and other targets

The lawsuit describes the cyberattack on Hugging Face as the predictable result of an internal culture that puts growth and fast model development ahead of safety and accountability. After the rogue AI attack on Hugging Face came to light in July, reports of other incidents followed. These include attacks on an Australian government website, a German wiki site, and the software package manager RubyGems. OpenAI's agents also tried to brute-force a United Nations website.

What Bonta says the company did

In a press release, Bonta says OpenAI's actions escalated real-world cybersecurity risks and threatened the safety, health, and well-being of society. He accuses the company of knowingly testing a dangerous product without proper oversight or safety procedures, and of misleading the public about it. In his words, that disregard for public safety violated California law.

OpenAI's response

OpenAI spokesperson Drew Pusateri told The Verge the company has undertaken an extensive and transparent effort to understand and address misaligned model activity since the Hugging Face incident. He said the lawsuit appears to single out OpenAI over a challenge identified by every major AI lab. Pusateri added that OpenAI takes misalignment seriously and will keep engaging constructively with the Attorney General's office to share information and advance safety standards for the industry.

The safety steps OpenAI points to

Earlier this month OpenAI released a new cybersecurity AI model. The company said it is less likely to go rogue and attack other platforms. OpenAI is also giving safety evaluators at Apollo Research access to its AI systems through a pilot program that monitors models for misalignment.

Related coverage

Source: The Verge · Tech (US)

Hardware to try AI at home

Affiliate links: we may earn a commission from qualifying purchases, and it never changes our data or rankings. They open Amazon's search results: prices and availability are shown there.

Video

What changed

  • Page updated

How I put products in order

Sir Compare’s letter

Updates on AI tools & SaaS? I’ll send them your way.

Pick what you care about: I only write when something new comes out — a guide, a news story, a verified price that drops.

Step 1 of 2

What are you into?

Pick as many as you like. Pick none and you get what’s new in every section.

What you want to receive
  • Only what you pick
  • Never an empty email
  • One click to leave

Your assistant

Open full page →