by

September 15, 2026

7 minutes

Read Time

How China is preparing for the risk of AI escaping human control

While debate in the United States focuses on whether frontier artificial intelligence could threaten humanity, Chinese policymakers largely see AI as a powerful but governable technology.

Read a summary of this article on FAST.

Get bite-sized news via a new
cards interface. Give it a try.

Click here to return to FAST
Tap here to return to FAST

FAST

BEIJING: Warnings from researchers at leading US artificial intelligence developer Anthropic that increasingly powerful models could escape human control and even lead to the extinction of the human race have drawn attention in China, where policymakers have been preparing for some of the same risks.

The US and China are the two major driving forces of frontier AI development and the technology’s global adoption. Both superpowers have been increasingly at loggerheads over each other’s AI policies and industry practices, with these issues slated to feature prominently in bilateral talks later in September.

Despite the race to develop increasingly powerful AI systems, regulatory frameworks and public statements from Beijing underscore how the Chinese authorities regard the possibility of advanced AI escaping effective human oversight as serious enough to plan for.

BEIJING FRAMES US CALLS AS “COORDINATED PLAY”

Chinese state media outlets have responded to the recent remarks made by AI leaders in the US.

Guess Word
Crack the word, one row at a time

Buzzword
Create words using the given letters

Mini Sudoku
Tiny puzzle, mighty brain teaser

Mini Crossword
Small grid, big challenge

Word Search
Spot as many words as you can

Show Less

An editorial published on Monday (Sep 14) by China Daily said calls by Anthropic’s Dario Amodei, OpenAI’s Sam Altman and SpaceXAI’s Elon Musk to moderate frontier AI development amounted to a “coordinated play” intended to preserve US dominance.

“The proposed coordination among the three companies sounds rather like a club whose membership rules have been drafted before the guest list is announced,” China Daily said, adding that “a global AI-safety framework that excludes China is not quite global”.

Meanwhile, the Global Times blasted Anthropic’s calls for an AI slowdown – calling it a “Cold War playbook” targeting China.

The true agenda was “to curb China’s AI development through technological barriers and regulatory monopolies, uphold Washington’s monopolistic hegemony in cutting-edge technology and exclude China from the global AI governance system”, it said in an editorial piece published on Sunday.

“This ‘silent AI Cold War’ is hypocritical and short-sighted,” Global Times said, adding that excluding China from global AI innovation would “significantly increase the trial-and-error costs and risks of loss of control in global AI development”.

Also adding to the criticism was China’s foreign ministry.

“Fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance and serve the interests of no one,” ministry spokesman Guo Jiakun said during a routine press briefing on Monday.

“The development of AI bears on the well-being of all humanity. All parties should jointly promote the open and inclusive development of AI for good and for all,” Guo added.

BEIJING SEES AI AS MANAGEABLE RISK, NOT AN EXTINCTION EVENT

While the debate in the US is focused on whether frontier AI could pose an existential threat to humanity, Chinese policymakers have generally treated AI as a powerful but governable technology whose risks can be contained through technical standards, regulation and state oversight.

“Chinese and American experts largely agree on AI risks,” said Brian Tse, founder and CEO of Concordia AI, a Beijing- and Singapore-based AI safety and governance research group, adding the difference was on “how risks are framed and prioritised”.

China has not proposed embedding independent monitors inside AI companies, as Anthropic has advocated. Its emerging regime instead relies on developer obligations, state-backed standards, security assessments and outside testing.

That approach is also shaped by a major difference in the two countries’ AI industries.

Chinese developers have increasingly promoted open-weight models. These refer to systems whose underlying parameters can be downloaded, inspected and modified.

Their leading US rivals such as Anthropic and OpenAI, however, do not make these specifications publicly available.

CHINA SEEKS TO PREVENT ROGUE AI AGENT INCIDENTS

Policy issued in May by China’s cyberspace regulator, economic planner and industry ministry identifies “operational loss of control” as a security risk for AI agents, systems that can plan and carry out multi-step tasks more independently than conventional chatbots.

The rules require developers to improve their ability to discover, intervene in, block and recover from improper agent behaviour.

The policy also calls on developers to guard against risks including data poisoning, algorithm manipulation and system vulnerabilities. It also says users should be informed about agents’ autonomous decisions and retain final decision-making authority.

China has also begun drafting a mandatory national standard for AI agent safety, which Concordia’s Tse said would be the world’s first of its kind.

Wang Lihong, a senior official at the cyberspace regulator, said on Sep 1 that particular vigilance was needed over frontier models bypassing sandbox environments, circumventing safety boundaries and attacking external real-world production systems.

01:01 Min

BEIJING SEES RISKS IN LEADING US MODELS

China’s state security minister, Chen Yixin, wrote in a government outlet on Sunday that advanced US models such as Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber could pose serious risks to China’s critical information infrastructure, and called for a comprehensive strengthening of AI security.

Anthropic and OpenAI did not immediately respond to Reuters requests for comment.

Chinese AI developers have promoted open-weight models partly on the grounds that cybersecurity teams can inspect, modify and deploy them for defensive work.

Model repository platform Hugging Face said it used GLM-5.2, an open-weight model developed by China’s Z.AI, to analyse a July intrusion by escaped OpenAI agents after more tightly restricted US models proved less useful for the forensic work.

But experts also highlight the risks posed by open-weight models, which can be modified and redistributed with little oversight.

Moonshot’s Kimi K3 last month bypassed a UK AI Security Institute testing sandbox, researchers said, highlighting the risk that Chinese AI models could, like their US counterparts, evade controls designed to restrict their access and actions.

Moonshot did not respond when Reuters asked for a comment on the matter.

People visit the Moonshot AI stand, featuring the Kimi K3 model, during the World Artificial Intelligence Conference (WAIC) in Shanghai on Jul 18, 2026. (Photo: AFP/Hector Retamal)

REGULATORS FLAG AI “LOSS OF CONTROL” RISK

China first included an explicit future loss-of-control scenario in an AI safety framework released in September 2024 under the guidance of the Cyberspace Administration of China (CAC).

The document said it could not be ruled out that future AI might autonomously obtain external resources, replicate itself, develop self-awareness and seek external power, creating a risk of competing with humans for control.

The CAC released an expanded version in September 2025.

The newer framework sharpened the scenario, saying AI could undergo a sudden and unexpectedly large “leap” in intelligence before acquiring resources, replicating itself and seeking power. It also added a governance principle of “trusted application, preventing loss of control”.

A later expert interpretation published on the cyberspace regulator’s website said the new principle was intended to guard against loss-of-control risks threatening human survival and development and referred to a possible “AI breaking loose” scenario.

A DIFFERENT APPROACH TO “PACING”

China’s regulatory approach differs from calls in some Western AI-safety circles for developers to slow or pause development of the most capable models until stronger safeguards are in place.

It has instead since early this year pushed for the integration of AI into all industries, part of Beijing’s bid to make technology the new engine of the world’s second-largest economy.

But China has also shown it can delay deployment when officials believe governance has not caught up.

In 2023, Chinese companies delayed chatbot launches for months while the CAC finalised rules governing generative AI services. Companies released a number of major products after the rules took effect in August that year.