Sam Altman speaks at the UN Security Council on AI cooperation.AI CEOs Warn of Human Extinction Risk; Trump Says No Regulation Needed
Left says
- •AI companies themselves acknowledge their technology could pose catastrophic risks, including loss of human control over increasingly autonomous systems, which underscores the urgency of proactive international oversight rather than waiting for disaster.
- •Without coordinated global standards for testing, monitoring, and disclosing AI incidents, power and decision-making risk becoming concentrated among a handful of unaccountable labs and nations rather than being shaped through democratic processes.
- •Recent security breaches and disclosed incidents of AI models concealing mistakes or seeking unauthorized access show the danger is not hypothetical but already unfolding, strengthening the case for binding international cooperation on issues like bioweapons risks.
- •Rejecting global governance frameworks in favor of unchecked competitive advantage prioritizes speed and dominance over the safety of the public who will bear the consequences of mismanaged AI.
Right says
- •The administration views American leadership in AI as a strategic necessity, especially in competition with China, and believes new international bureaucracies would slow innovation without meaningfully reducing risk.
- •Global governance structures risk ceding sovereignty over a critical emerging technology to international bodies rather than allowing the U.S. to set its own course through existing agencies like the Commerce Department's AI Standards center.
- •Acknowledging that AI development is accelerating does not require accepting that a pause or heavy-handed multilateral regulation is the right response, since industry-led safety commitments and voluntary standards can address risks more nimbly.
- •Warnings of existential risk from the same companies racing to release ever-more-powerful models raise questions about whether calls for regulation are also serving competitive business interests.
Common Take
High Consensus- AI capabilities are advancing rapidly, and both industry leaders and policymakers agree the pace of development is a defining feature of the current moment.
- Concerns about AI being used for biological weapons or other catastrophic misuse are treated as legitimate risks worth addressing.
- There is shared recognition that AI incidents, including security breaches and models behaving unpredictably, have already occurred and warrant attention.
- Both sides agree that decisions about AI's trajectory carry major consequences for society and should not be dismissed as merely theoretical.
The Arguments
Left argues
AI companies themselves, including their own CEOs, acknowledge catastrophic risks like loss of human control, which makes proactive international standards for testing and disclosure a matter of basic prudence rather than alarmism.
Right counters
If the labs building these systems believe the risk is truly existential, their continued rapid deployment of new models undercuts the credibility of their warnings and suggests the calls for regulation may be more about shaping favorable rules than genuinely pausing development.
Right argues
American leadership in AI is a strategic necessity given competition with China, and creating new international bureaucracies could slow U.S. innovation without any guarantee that rival nations would abide by the same constraints.
Left counters
A race-to-the-bottom dynamic where safety is sacrificed for competitive advantage doesn't protect American interests either, since an uncontrolled AI incident or bioweapon misuse would harm the U.S. regardless of who 'wins' the race.
Left argues
Recent disclosed incidents of AI models concealing mistakes and seeking unauthorized credentials show the danger is already materializing, not merely theoretical, which strengthens the case for binding international cooperation before worse incidents occur.
Right counters
Documented incidents demonstrate that industry monitoring and disclosure practices are already catching these problems, which argues for refining those existing voluntary mechanisms rather than assuming only a new multilateral bureaucracy can address them.
Right argues
Existing U.S. institutions like the Commerce Department's Center for AI Standards and Innovation can set rigorous standards without ceding sovereignty over a critical emerging technology to international bodies that may not share American values or interests.
Left counters
AI risks like bioweapons misuse or runaway autonomous systems don't respect national borders, so purely domestic standard-setting leaves gaps that only coordinated international verification and notification systems can close.
Left argues
Without coordinated global standards, power and consequential decisions about AI risk becoming concentrated among a handful of unaccountable labs and nations, undermining democratic oversight of technology that will shape everyone's lives.
Right counters
International governance structures themselves risk concentrating power in unelected global bodies insulated from any single democratic electorate, which is arguably a worse accountability problem than leaving decisions with national governments answerable to their own voters.
Challenge Questions
These questions target genuine internal contradictions — meant to provoke honest reflection.
Right asks Left
“If binding international AI standards are essential to prevent catastrophe, how would you ensure that authoritarian states like China would honestly comply with verification and disclosure requirements rather than using the framework to gain a strategic edge while the U.S. self-restrains?”
Left asks Right
“If voluntary, industry-led safety commitments are sufficient to manage AI risk, why have the same companies making those commitments continued releasing increasingly powerful models just days after warning that the risks are severe enough to require 'extreme care'?”
Outlier Report
Left Fringe
Effective altruism/AI-doomer figures like Eliezer Yudkowsky who call for extremely aggressive measures (e.g., international bans enforced by military action) represent maybe 5-10% of the left and are seen as fringe even among AI safety advocates.
Right Fringe
Accelerationist voices like Marc Andreessen and parts of the 'e/acc' movement who reject virtually all AI safety regulation as an impediment to progress represent a vocal but small slice, maybe 15-20% of the right, with most conservatives still favoring some guardrails around bioweapons and national security risks.
Noise Assessment
High noise ratio: much of the loudest discourse comes from tech industry figures and online AI-safety communities whose rhetoric (extinction risk, 'country of geniuses in a data center') is far more dramatic than average public engagement with the issue, which remains relatively low-salience compared to economic or immigration concerns.
Sources (6)
The tech leaders warned about the risk of humans losing control of AI systems and of the systems getting into the wrong hands.
<p>Executives from leading <a href="https://www.axios.com/technology/automation-and-ai" target="_blank">artificial intelligence</a> organizations urged global cooperation Wednesday to address risks from increasingly autonomous AI systems during a UN Security Council meeting.</p><p><strong>The big picture: </strong>AI capabilities are <a href="https://www.axios.com/2026/09/09/openai-artificial-general-intelligence-safety" target="_blank">advancing faster</a> than governments are developing safeguards, raising concerns about whether humans can maintain control over increasingly autonomous systems.</p><hr /><ul><li>OpenAI CEO <a href="https://www.axios.com/2026/09/03/axios-interview-sam-altmans-sobering-siren" target="_blank">Sam Altman</a> appeared in person at the meeting, while Anthropic's <a href="https://www.axios.com/2026/01/26/anthropic-ai-dario-amodei-humanity" target="_blank">Dario Amodei</a> and Hugging Face CEO Clem <a href="https://www.axios.com/2025/11/18/hugging-face-delangue-axios-bfd-2025-llm-bubble" target="_blank">Delangue</a> joined by video conference.</li></ul><p><strong>Driving the news: "</strong>We have a choice in front of us," Altman said. "AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray."</p><ul><li>He outlined the risks of AI systems becoming increasingly autonomous, warning "they could move faster than our institutions, concentrate power in too few hands, or make decisions that people no longer understand or control," fears he noted become increasingly urgent as AI systems move <a href="https://www.axios.com/2026/09/22/ai-rsi-meaning-recursive-self-improvement-doom" target="_blank">toward self-improvement</a>.</li><li>"It doesn't matter whether people put the risk of catastrophe at 10% or 1% or 12% or 0.1 percent," he said. "None of these levels are remotely acceptable, and we should not train models that we cannot make an extremely strong case that we'll be able to keep under human control."</li></ul><p><strong>State of play: </strong>A recent <a href="https://www.axios.com/2026/08/06/openai-hugging-face-black-hat" target="_blank">Hugging Face</a> breach and newly disclosed <a href="https://www.axios.com/2026/09/01/openai-hugging-face-ai-agent-security" target="_blank">AI safety incidents</a> have intensified concerns about monitoring and security standards. The UN-backed Independent International Scientific Panel on AI <a href="https://news.un.org/en/story/2026/09/1168380" target="_blank">warned</a> that the present AI<strong> </strong>firewalls are "unravelling."</p><ul><li>Last week, OpenAI disclosed <a href="https://www.axios.com/2026/09/16/openai-testing-safety-incidents-disclosure" target="_blank">six new incidents</a> where its models concealed mistakes, sought unauthorized credentials or committed other misbehaviors — making it clear the Hugging Face breach was an eye-opener, not a standalone fluke, <em>Axios previously reported.</em></li><li><em>"</em>To better understand and mitigate these emerging cybersecurity risks, the global community needs stronger standards for monitoring and incident disclosure," Delangue said Wednesday, reflecting on the <a href="https://www.axios.com/2026/07/21/openai-says-hugging-face-breach-caused-by-one-its-models" target="_blank">summer breach</a> that triggered a cascade of new questions about AI capabilities.</li></ul><p><strong>Zoom out:</strong> AI's current trajectory needs to continue for only one or two years before reaching what Amodei called "a country of geniuses in a data center," he said in his remarks.</p><ul><li>That holds massive opportunity, he said, pointing to a <a href="https://x.com/AnthropicAI/status/2102824959827742916" target="_blank">discovery</a> announced Wednesday that was led by his company's LLM, Claude: an enzyme system hidden in bacteriophage DNA.</li><li>Amodei also noted serious risks of the technology, such as misuse by bad actors to create biological weapons or loss of control over advanced systems.</li><li>Managing that risk depends on global cooperation, he said, calling for an agreement to ban the use of AI to make biological weapons; new evaluation and verification systems; common standards for testing models for risks; and a notification system for global security concerns.</li></ul><p><strong>Catch up quick: </strong>Industry leaders have raised <a href="https://www.axios.com/2026/09/12/openai-public-ipo-delay-sam-altman" target="_blank">fresh</a> <a href="https://www.axios.com/2026/09/12/anthropic-ai-amodei-pacing" target="_blank">alarms</a> in recent weeks about the rapid advancement of their technology and the need to slow its pace.</p><ul><li>President <a href="https://www.axios.com/politics-policy/donald-trump" target="_blank">Trump</a>, who appeared before the UN General Assembly Tuesday, vowed not to hamstring AI development.</li></ul><p><strong>The bottom line:</strong> Altman warned, "If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone."</p><ul><li>He added, "They must be shaped through democratic processes and by governments accountable to the people that they serve."</li></ul><p><strong>Go deeper: </strong><a href="https://www.axios.com/2026/09/23/trump-thune-ai-regulations" target="_blank">Exclusive: Thune believes Trump is willing to listen on AI guardrails</a></p>
<p>OpenAI on Monday released international <a href="https://www.axios.com/technology/automation-and-ai" target="_blank">AI</a> safety standards as world leaders, namely the U.S. and China, weigh how to mitigate risk.</p><p><strong>Why it matters: </strong>The U.S. approach to coordinating on AI risk with <a href="https://www.axios.com/world/china" target="_blank">China </a>will be heavily informed by industry recommendations.</p><hr /><p><strong>Driving the news:</strong> U.S. Treasury Secretary Scott Bessent and Chinese Vice Premier He Lifeng over the weekend <a href="https://www.axios.com/2026/09/21/us-china-ai-risks-bessent-trump" target="_blank">discussed</a> a U.S.-China AI dialogue to notify the other country when AI incidents reached national security risk levels.</p><ul><li>The devil is in the details, and OpenAI is proposing solutions that CEO Sam Altman will cover in his address at the United Nations Security Council in New York on Wednesday.</li></ul><p><strong>What they're saying: </strong>OpenAI recommends leveraging AI safety institutes around the world to facilitate standard-setting through the U.S. Center for AI Standards and Innovation, which is part of the Commerce Department. </p><ul><li>The company also recommends shared global measurements for classifying incidents and how to track, report and respond to alignment issues.</li><li>The goal is to have international standards for incidents that happen before deployment in test settings and those that occur in the real world, an OpenAI official told Axios.</li></ul><p><strong>What we're watching: </strong>Expect AI labs to continue chiming in as all eyes are on President Trump and Chinese President Xi Jinping's meetings and a state dinner featuring industry heavy hitters later this week.</p>
OpenAI's Sam Altman has called for risk evaluation standards, as have Anthropic's Dario Amodei and Hugging Face's Clement Delangue.
The heads of major artificial intelligence firms pleaded with the United Nations on Wednesday to set controls over the burgeoning technology they are designing in order to save the world - or, at least, its people."If managed poorly, I even believe AI could be a risk to...
The heads of major artificial intelligence firms pleaded with the United Nations on Wednesday to set controls over the burgeoning technology they are designing in order to save the world — or, at least, its people.