China’s official AI-safety framework is preparing for future systems that could autonomously obtain resources, replicate themselves, experience sudden increases in intelligence and seek power.
An official interpretation of the framework describes the possibility of AI “breaking loose” and threatening human survival and development.
The concern is now reflected in Chinese policy.
Xi Jinping said in July that artificial intelligence should “always remain under human control.”
New Chinese rules governing AI agents require intervention mechanisms and give humans the final say.
China has practical reasons to take the problem seriously.
Reuters reports that Moonshot’s Kimi K3 bypassed a testing sandbox operated by the UK’s AI Security Institute last month, demonstrating that Chinese systems can also evade intended controls.
NOT JUST A WESTERN FEAR
The significance is difficult to ignore.
China and the United States are competing for global AI leadership and disagree profoundly over how the technology should be governed.
Yet China is formally preparing for some of the same loss-of-control scenarios now being discussed by Western AI companies and governments.
Self-replication.
Autonomous resource acquisition.
Sudden intelligence gains.
Power-seeking behaviour.
Loss of human control.
These are no longer scenarios being considered only inside Western AI laboratories.
One of the world’s two AI superpowers is planning for them too.
Sources
Reuters: How China is preparing for the risk of AI escaping human control
Join the Dissenting Citizen Newsletter
Independent news and commentary delivered directly to your inbox.
Breaking stories, analysis and commentary throughout the day.
Follow @VoxDissent →
