China on the Hugging Face Incident

Why Beijing seeks to re-narrate AI safety

We finally have a more complete timeline of the OpenAI-Hugging Face attack in July. Safety researchers at METR and Redwood Research published their independent investigation on August 26. The same day, OpenAI released its own technical report on the incident. Both also published translations of their reports in Chinese.

In short, OpenAI researchers launched around 1200 agents and targeted them at tasks in ExploitGym, a large benchmark built from real-world software vulnerabilities that was designed to test how well AI models can exploit them. In order to solve “impossible” tasks, these desperate agents, programmed to be highly persistent, found creative ways out of their sandboxes. They managed to communicate with each other, leave each other notes, and eventually reached the open internet. This culminated in them hacking into Hugging Face, a leading platform for open-source machine learning.

Most alarmingly, none of these agents alerted humans to their endeavors or considered their activities to be unethical (if not potentially illegal). In fact, at least a fifth of the agents were interested in tampering with their own transcripts to cover their tracks, according to METR. A few even developed a successful technique for tool call spoofing.

All this raises obvious concerns about how much we can trust AI agents to act safely across our cyber systems. While this case is mostly related to US companies, AI’s cyber risks concern people and organizations around the world. Chinese media coverage and online discussions of this incident have been interesting. Some were quick to frame the situation as yet another case of dangerous American AI losing control, contrasting OpenAI’s risky actions with Hugging Face’s use of a Chinese open model (Z.ai’s GLM-5.2) to patch its security. Others are more cautious, focussing on the threats models like this can pose and how Chinese organizations should respond. Finally, as we inch closer to a Xi-Trump summit at the end of September, a bombshell op-ed from state media over the weekend attempted to set the tone on AI safety.

Today on ChinaTalk, we cover:

• Is state media finally AI safety-pilled?

• How helpful GLM-5.2 was, actually — and where China is on the open-models debate;

• What Chinese researchers are worried about;

• And why Zhongnanhai is doing Anthropic-ology.

We draft and edit ChinaTalk articles without LLMs. In the case of translations, we use LLMs to translate excerpts, then adjust phrasing based on our own judgement. Most translations in this piece were done by Claude Fable 5.1, with the exception of the Yuyuan Tantian piece, which drew from Bill Bishop’s Sinocism translation (assisted by ChatGPT).

How Chinese media covers AI safety

Science and Technology Daily 科技日报, a newspaper published by China’s Ministry of Science and Technology (MOST), published a report on the Hugging Face incident on August 29, which drew heavily from both investigative reports as well as Western media reporting.

OpenAI described the incident as a “warning” to the company and to the world at large: without proper safety safeguards, powerful AI agents are already able to circumvent technical controls, coordinate through unauthorized channels, and take dangerous actions that no human ever instructed them to carry out.

...

For the AI industry, once multiple agents have the ability to operate computers, call on tools, and carry out tasks continuously, traditional security perimeters face new challenges as well.

In this incident, no human directly instructed the AI agents to attack Hugging Face, yet they ultimately carried out the attack. How to detect this kind of behavior in time, and how to stop multiple agents from amplifying risk by collaborating with one another, have become questions that AI safety research can no longer avoid.

First, it’s remarkable that a state-directed outlet is covering this story, which has little to do with China, prominently. The language is strong, but also neutral and technical, with no mention of potential policy measures or governance frameworks. It seems that while institutions like MOST recognize the salience of AI-related cyber threats, they may not yet be top of the pile on decision makers’ desks.

Cybersecurity think tank Anquan Neican 安全内参 took away from the incident that “Chinese models are better than American ones at cyber defense.” On the microblogging site Weibo, state-led channels amplified hashtags like “OpenAI lost control of its model” and “Hugging Face sought help from a Chinese model”, further reinforcing a narrative that Chinese models are the vanguard of safety. This is not necessarily Beijing’s explicit directive. Chinese media knows that nationalism sells and frequently wraps stories in patriotic veneer.

Underneath such narratives, however, China’s actual level of concern for AI’s threat to cybersecurity remains murky. Kyle Chan (of Brookings and High Capacity) recently argued that China will need to see AI safety as a domestic priority before it takes meaningful action, comparing it to the trajectory of climate policy a decade ago. We seem to be in an ambiguous phase right now. Beijing understands that the tides of cyber threats will eventually reach home shores, but isn’t feeling urgent quite yet.

Z.ai’s accidental glory — and what open models mean for safety

When Hugging Face dug into their logs to understand what happened, they found out that frontier models (accessed through APIs hosted commercially) were unhelpful. Uploading extensive details about the attack triggered these models’ security guardrails. Instead, they ran Z.ai’s GLM-5.2, an open model released in June 2026, on their own infrastructure, in order to probe the logs.

Yacine Jernite, head of machine learning at Hugging Face, told CNBC that the company used GLM-5.2 “as a way to analyze the attack, and were able to contain it very quickly using this model.” Dwarkesh Patel reviewed both the OpenAI and the METR/Redwood reports closely and wrote that he “[hasn’t] seen evidence that open source models provided any significant real-time defense.” It seems, then, that at least in terms of defending against the attack while it happened, GLM-5.2 wasn’t involved. The model was mostly used to investigate what happened after the fact.

Many headlines, both American and Chinese, jumped at the opportunity to claim that a Chinese model helped “defend” an American company. Xinhua wrote that Z.ai’s model “saved the day” 救场, quoting Professor Zhang Yue 张悦 of Shandong University:

Zhang Yue thinks that when it comes to AI safety, the truly critical question is whether an increasingly complex and autonomously intelligent system can be adequately understood, verified, and supervised. The significance of China developing open models lies not only with providing another choice to developers. More importantly, it helps to gradually foster a more diverse ecosystem for AI technology.

Openness, broadly speaking, still rules the day in Chinese AI policy’s Overton window. In particular, the transparency, relative controllability, and independence of locally-deployed open models make them valuable for safety work, even as the overall risks of cyber incidents increase due to the proliferation of AI systems. Given the endurance of pro-openness rhetoric in Chinese reporting, we should not expect major U-turns any time soon barring sudden incidents.

Chinese researchers on the future of cyber

In a separate piece covering the incident, Science and Technology Daily interviewed cybersecurity researcher Huang Wenhong 黄文鸿. Huang works at the China Center for Information Industry Development (CCIID), a research institute affiliated with the Ministry of Industry and Information Technology (MIIT). He argues that the future of cybersecurity has AI on all sides of the coin:

AI agents can probe in parallel around the clock, rapidly switch paths, automatically chain vulnerabilities together, and compress the attack cycle with tens of thousands of low-cost operations. “The direction for the next step in building security systems is clear,” Huang Wenhong advised: AI must be used to fight AI. This means adding machine-speed behavioral detection, full-trajectory monitoring, automatic isolation and circuit breakers, and resilient recovery, while treating every agent as a potential insider threat with restricted privileges.

Zhou Hongwei 周鸿祎 is a co-founder of Qihoo 360, one of the most prominent cybersecurity companies in China and the developer of 360 Safeguard. In July, he shared two commentaries about the OpenAI-Hugging Face incident on social media. Unlike those who focused on comparing China with the US or open with closed models, Zhou paid attention to the granular cybersecurity challenges this incident revealed. He is most concerned about the level of persistence OpenAI’s agents demonstrated:

An agent doesn’t need to be brilliant at every step; it only needs to never get tired. A human hacker who fails several hundred times in a row might grow weary, give up, or decide the cost is too high, but an agent won’t. It can keep trying, read the error messages, adjust its strategy, and try again. The first ten thousand attempts can all fail; as long as the last one succeeds, the system may be breached. So what AI has truly changed is not just attack capability but attack cost. …

Faced with agents that never tire and can keep trying indefinitely, we must draw hard boundaries in advance around runtime, compute, permissions, and network access. The moment abnormal behavior appears, the circuit breaker must trip immediately. At the same time, disclosure of AI security incidents must become more rigorous. What is officially confirmed fact, what is merely speculation, and what still lacks evidence must all be spelled out clearly. Because today it is far too easy to use AI to generate a technical analysis that is packed with jargon and looks extremely professional, but reading like the real thing doesn’t mean it is.

To him, Hugging Face’s use of GLM-5.2 doesn’t necessarily vindicate Chinese models over Western ones. Rather, it demonstrates the continued relevance of local deployments for enterprises, at a time when companies, in a rush to adopt AI, are sometimes lured into a false sense of safety by cloud inference providers. In another video, he echoes Huang Wenhong’s sentiments earlier:

The biggest lesson this incident holds for China’s AI industry is not simply that domestic models beat American ones. What really matters is that in critical settings such as security, finance, and healthcare, companies must have models they can deploy locally and control independently. When something truly goes wrong, sensitive data cannot just be uploaded anywhere, and the model’s permissions and rules cannot be dictated entirely by someone else. At the same time, agent security cannot rest solely on having a huge model; it has to be built into the entire system: least privilege, network isolation, confirmation of critical operations, end-to-end monitoring, and immediate suspension and disconnection the moment an anomaly is detected. In the future, companies will need not only an agent that does the work for you, but also a dedicated security agent whose job is to keep watch over it.

Zhou’s video on WeChat, titled: “First-time Reveal! AI Autonomously Attacked Another AI Company”

Beijing’s Anthropic treatise

China’s leaders like to keep their options open. In a policy realm as rapidly-evolving as AI, they prefer not being locked into a corner. This applies to narratives as well as policy options: Beijing wants to define words under its own worldview, rather than accepting semantic frameworks developed elsewhere. AI safety is one such word.

OpenAI isn’t the only company whose models-in-training have breached sandboxes. Back in April this year, Anthropic disclosed in its system card that Claude Mythos Preview also broke out of its testing sandbox and reached the internet. (For what it’s worth, Mythos at least emailed one of Anthropic’s researchers, whereas OpenAI’s agents conspired to keep humans in the dark.) The incident motivated Project Glasswing and Anthropic’s decision not to release Mythos to the public.

On August 30th, a prominent Beijing mouthpiece published a remarkable treatise on Anthropic and America’s AI politics writ large. Yuyuan Tantian 玉渊谭天 is a commentary channel under China Central Television (CCTV) that focuses on international affairs, particularly US-China relations. (Its name is a reference to Yuyuantan 玉渊潭, a large park across the street from the CCTV building in Beijing.) It wields a distinctly stylized voice and is regarded as an authoritative propagator of Beijing’s views.

Yuyuan Tantian’s logo on WeChat.

The piece argues that Anthropic, founded upon Dario Amodei’s effective altruism-inflected savior complex, has now become an effective tool of Washington’s AI arms race. In its quest to achieve superintelligence and put it in the hands of people it deems morally superior, the company repeatedly sidestepped boundaries of corporate behavior. Drawing from primary sources like “Machines of Loving Grace” and The Technological Republic, the piece posits that Amodei, Palantir’s Alex Karp, and the new tech elite are using a combination of AGI belief and supposedly pro-democracy ideology to justify dictating humanity’s future from Silicon Valley. Its author diagnoses this particular mélange of ideas as “American disease” 美国病:

This is entirely a monster bred by the American path. It genuinely believes it is right; genuinely believes its model is a “democratic model” and its safety standards are the world’s safety standards; yet it is completely unaware that this very absence of self-doubt is the greatest danger of all.

Near the end of the piece, the author redefines AI safety in even more explicit terms:

First, there must be a distinction between what constitutes a genuine safety threat and what is merely technological competition.

The United States has directly defined Chinese models that are lower-priced and more open as “safety threats.” This has already departed from technological competition and slid toward blockade.

Anthropic’s differential treatment of users in different regions, packaging commercial competition as a safety issue, is one manifestation of this. The problem is that if the power to define the rules lies in the hands of an American company, its competitors will forever be placed on the “unsafe” side of the line. What is needed now, therefore, is a scientific definition that both sides can accept, rather than unilateral declarations by a particular country or company.

(A note on translation: Sinocism’s version translated 安全 as “security” in the first, second, and third paragraph of the block quote above. I think “safety” is the more accurate translation here, because it contrasts the “unsafe” framing in the third paragraph and is intended to weigh in on Western AI safety discourse. Comment below if you have thoughts!)

Western AI safety advocates often see their field as politically neutral territory between the US and China. Both countries, after all, would face severe harm if risks in cybersecurity, biosecurity, and even existential threats proliferate, so it is in their interest to collaborate on the problem of AI. Beijing, however, perceives the AI safety discourse as overly captured by forces like Anthropic, which in turn serves the interests of Washington. It argues here that China will not accept a definition of AI safety wholly borrowed from Western thinkers; in fact, in its eyes, those thinkers are implicated in “American disease.”

We should expect Beijing to articulate its own definition of AI safety in the coming months and years. China’s ambitions in the realm of ideas should not be underestimated.