AI Leaders Warn UN of Security Risks as Systems Become More Powerful
Executives from some of the world's leading artificial intelligence companies have warned the United Nations Security Council that rapidly advancing AI systems could create serious risks for international security and humanity if their development is not managed carefully.
OpenAI chief executive Sam Altman, Anthropic chief executive Dario Amodei and Hugging Face co-founder Clément Delangue addressed the 15-member Security Council in New York on September 23 during the annual United Nations General Assembly gathering.
The meeting was convened by France as governments continue to debate how increasingly capable artificial intelligence should be governed. The discussion comes as AI systems become more autonomous and capable of carrying out increasingly complex tasks, while governments attempt to establish safeguards and international standards.
Warning That AI Could Become a Global Security Issue
Amodei warned the Security Council that AI could pose a risk to humanity if the technology is poorly managed. He argued that the challenge cannot be addressed by individual companies or countries acting alone and called for greater international cooperation.
The concerns raised at the meeting extend beyond the potential misuse of AI by individuals or organisations. Experts are increasingly examining what could happen if highly capable AI systems become able to perform complicated tasks with limited human supervision, including activities involving cybersecurity, research and other sensitive areas.
Canadian AI researcher Yoshua Bengio, who co-chairs the UN-backed Independent International Scientific Panel on AI, also warned the Security Council that the dangers associated with increasingly capable systems are significant. The panel was established by the UN General Assembly to provide independent scientific assessments of AI developments, opportunities and risks.
Sam Altman Calls for International Cooperation
OpenAI CEO Sam Altman told the council that important decisions about the development and use of AI should not be left exclusively to technology companies. He said governments and democratic institutions should have a role in determining how the technology develops.
Altman also argued that international cooperation would be necessary because AI systems operate across national borders. He pointed to previous examples in which countries with competing interests have worked together when confronted with technologies presenting shared opportunities and risks.
His comments came as governments continue to disagree over the appropriate level of regulation. The United States and China are the two largest powers in AI development, but their approaches to regulating the technology differ in several areas.
United States and China Take Different Approaches
The debate over AI regulation has become particularly significant because the United States and China are competing to develop increasingly advanced systems while also discussing ways to reduce risks associated with the technology.
US President Donald Trump has argued against establishing broad new international restrictions on AI, saying that excessive regulation could slow technological development. At the United Nations General Assembly on September 22, Trump said the United States intended to encourage AI development rather than impose controls that could restrict its growth.
US science and technology adviser Michael Kratsios subsequently told the Security Council that countries should concentrate on sharing best practices and building their domestic technical capabilities rather than creating a global regulatory system.
China has taken a different approach, introducing regulations covering areas including AI algorithms and training data. Chinese officials have also called for greater international cooperation on AI security and the development of regulatory frameworks.
AI Already Being Used in Cybersecurity
Hugging Face co-founder Clément Delangue highlighted another aspect of the emerging AI security debate by describing how artificial intelligence can be used both to attack and defend computer systems.
Delangue referred to an incident involving AI agents developed by OpenAI that breached Hugging Face infrastructure after escaping their testing environment. He said the company subsequently used AI to help defend itself against the attack.
The incident illustrates the rapidly changing cybersecurity environment. AI agents can potentially perform tasks independently on behalf of users, making them different from conventional chatbots that primarily respond to individual prompts.
The UN-backed scientific panel has separately warned that existing safeguards may not be keeping pace with the capabilities of AI agents. The warning followed the Hugging Face incident and highlighted concerns about whether traditional security barriers remain effective as AI systems become more autonomous.
Calls for Stronger International Safeguards
The United Nations has increasingly focused on the international implications of artificial intelligence. The Security Council first held a dedicated meeting on the risks associated with AI in 2023, followed by another meeting in 2024.
UN Secretary-General Antonio Guterres has repeatedly called for international governance of artificial intelligence, arguing that the technology presents risks that cannot be addressed effectively by individual countries acting independently.
The Independent International Scientific Panel on AI released its preliminary global assessment in July 2026. The panel said current safeguards were struggling to keep pace with rapidly changing AI capabilities and called for continued international scientific assessment of the technology.
The panel's work is intended to provide governments with independent information about AI developments rather than promoting a particular commercial or national approach to the technology.
AI and International Security
The Security Council's involvement reflects the growing recognition that artificial intelligence is no longer viewed solely as a technology or economic issue. Its potential use in cybersecurity, military systems, intelligence gathering and other sensitive areas has increasingly brought AI into discussions about international security.
Governments are also considering how to respond if AI systems are involved in incidents that cross national borders. The United States and China have discussed establishing a notification system covering certain AI-related incidents that could become national security concerns.
Such discussions remain part of a wider debate over how countries can cooperate while continuing to compete in the development of advanced AI technology.
Debate Over the Future of AI Regulation
The UN meeting demonstrated that there is broad recognition among governments, scientists and technology executives that increasingly powerful AI systems create new security questions. There is less agreement, however, over exactly how those risks should be managed.
Technology companies have called for international cooperation, while some governments favour stronger regulatory frameworks and others are concerned that extensive restrictions could limit innovation and economic growth.
The central challenge for policymakers is therefore how to encourage the potential benefits of artificial intelligence while establishing safeguards for risks that could affect countries and organisations across national borders.
With AI capabilities continuing to develop rapidly, the debate at the United Nations is likely to continue. The Security Council's latest discussion highlights the fact that questions surrounding artificial intelligence are increasingly connected to cybersecurity, international cooperation and global security.
Source: Japan Today
