Skip to content
Sir CompareComparisons with evidenceYouTubeTelegram
United States · USD

News

Open source AI under attack: what Hugging Face is changing

Updated

Illustrative photo: AI Tools & SaaS: Code on computer monitor (Unsplash)
Illustrative photo

Why open platforms attract attackers

Irene Solaiman, chief policy officer and SVP of strategy at Hugging Face, said the same traits that make open AI platforms useful also make them easier to attack. She spoke on the Equity podcast. Open doors and rich data draw both contributors and intruders, she said. In a separate conversation with TechCrunch she called the mix a reason the company is a prime target. Her worry is that the openness that makes the community special is also what makes it targeted.

The attacks cited: OpenCode, OpenAI's swarm, Fireworks

Solaiman pointed to the October attack on OpenCode, which went after users' third-party API keys and caused financial damage. Before that, this summer an OpenAI agent swarm breached Hugging Face, apparently trying to cover its tracks after reaching answers to cybersecurity evaluations. The risks go beyond community platforms.

On Monday, AI infrastructure startup Fireworks said attackers had accessed some customers' model weights, datasets and billing information. Fireworks temporarily disabled certain self-service dataset functions while it investigates. Solaiman said the OpenCode target was its third-party API keys, while Hugging Face drew the swarm because of the evaluation datasets it holds.

What a pullback could look like

Solaiman raised the danger of open source closing down at a time of broad public support for open models. Much of the industry has backed open AI models, including Nvidia's Jensen Huang, and governments, enterprises and startups increasingly build on them. A pullback could mean limits on who can upload models or datasets, stricter identity checks, or extra vetting before developers reach certain features. Those steps could shield platforms from attackers. They could also make it harder for independent researchers and hobbyists to join communities that once welcomed them. Hugging Face is trying to avoid that path for now.

The token and network changes at Hugging Face

Solaiman said Hugging Face has been strengthening monitoring tools to catch hacks and taking steps to prevent them. She noted that monitoring is expensive to maintain and not every platform can afford it. On access, she said administrative access tokens helped the agent swarm get further in, so the company now rotates them far more often. Hugging Face also added stricter network controls. In practice, credentials that give admins broad access to its systems now change more frequently, so a stolen token stays useful for less time.

GitHub's parallel tightening

Other parts of the open source ecosystem face similar security trade-offs. GitHub, the largest host for open source projects, has been fighting attackers who inject malicious code into projects to spread malware quickly. The company began tightening security requirements for publishing packages to the npm registry. It also started revoking older authentication tokens and limiting how long new ones stay valid. Zach Steindler, a GitHub engineer working on software supply chain security, is cited in the report. The pattern is the same: shorter-lived credentials and stricter publishing rules.

Related coverage

Source: TechCrunch · AI

Hardware to try AI at home

Affiliate links: we may earn a commission from qualifying purchases, and it never changes our data or rankings. They open Amazon's search results: prices and availability are shown there.

Video

What changed

  • Page updated

How I put products in order

Sir Compare’s letter

Updates on AI tools & SaaS? I’ll send them your way.

Pick what you care about: I only write when something new comes out — a guide, a news story, a verified price that drops.

Step 1 of 2

What are you into?

Pick as many as you like. Pick none and you get what’s new in every section.

What you want to receive
  • Only what you pick
  • Never an empty email
  • One click to leave

Your assistant

Open full page →