Anthropic Researchers Warn AI Could Kill Everyone This Decade
Left says
- •Warnings from insiders at the companies building this technology carry special weight because these researchers have access to unreleased, more advanced models the public hasn't seen.
- •The race between AI labs is driven by commercial incentives and competitive pressure to be first, even as insiders privately fear the consequences of moving too fast.
- •This moment underscores the need for meaningful government regulation and possibly international treaties, since companies alone are unlikely to slow down without external pressure.
- •The fact that senior researchers grow more concerned with seniority and access to information suggests these fears reflect informed risk assessment rather than sensationalism.
Right says
- •Skepticism is warranted given that these same companies are racing toward IPOs and stand to benefit financially from hype that could also invite regulation favoring dominant incumbents over new competitors.
- •No researcher has offered concrete evidence or a specific mechanism for how an AI apocalypse would actually unfold, leaving these warnings as speculative gut feelings rather than scientific predictions.
- •Heavy-handed regulation could hand control of AI development to a handful of powerful firms and slow American innovation while rivals like China continue advancing their own systems unchecked.
- •Government overreach in the name of safety has a history of unintended consequences, and lawmakers should be cautious about legislating based on vague, unverifiable fears.
Common Take
High Consensus- Multiple senior researchers at Anthropic, including Jacob Coxon and Evan Hubinger, have publicly stated they believe AI poses a genuine existential risk within the next decade.
- Current AI systems like Claude, Grok and ChatGPT are not considered capable of causing human extinction today.
- There is no scientific method to calculate precise odds of an AI-driven catastrophe, and estimates among experts vary widely, from under 1% to nearly 100%.
- Both AI companies and outside observers agree that competitive pressure between AI labs is a central factor driving the pace of development.
The Arguments
Left argues
Insiders with access to unreleased, more advanced models are uniquely positioned to assess risk, and the pattern that more senior, better-informed researchers express greater concern suggests these fears stem from genuine risk assessment rather than public posturing.
Right counters
These same insiders work for companies racing toward IPOs and stand to gain from hype that inflates their valuations and could invite regulation that entrenches them as dominant incumbents, so their access to secret models doesn't erase the conflict of interest.
Right argues
None of the researchers, including Hubinger himself, offered a concrete mechanism for how an AI-driven extinction would actually occur, leaving these warnings as unfalsifiable gut estimates rather than scientific predictions that should drive policy.
Left counters
The absence of a precise mechanism doesn't make the risk illegitimate — novel catastrophic risks like pandemics or nuclear escalation were also warned about in probabilistic terms before their exact pathways were fully understood, and waiting for certainty before acting could mean waiting until it's too late.
Right argues
Heavy-handed regulation driven by vague, unverifiable fears could hand control of AI development to a small number of powerful incumbents, stifle American innovation, and cede ground to China and other rivals who will keep advancing regardless of U.S. rules.
Left counters
The researchers themselves are describing an industry-wide race dynamic where no single company can unilaterally slow down without ceding advantage to competitors, which is precisely the collective-action problem that only external regulation or international coordination can solve.
Left argues
The fact that companies are simultaneously racing ahead commercially while their own top safety researchers admit there's no plan to solve alignment for superintelligence reveals a dangerous gap between private fear and public action that only outside pressure can close.
Right counters
If executives truly believed catastrophe was imminent, the rational business move would be to halt development entirely rather than push toward IPOs, suggesting the public warnings may be more about shaping the regulatory and reputational environment than reflecting operational certainty.
Left argues
This moment illustrates why voluntary self-regulation is insufficient — companies are explicitly saying they feel trapped in a race they can't exit alone, which is exactly the kind of market failure that government intervention or international treaties exist to correct.
Right counters
Governments have a poor track record of regulating fast-moving technology wisely, and hastily crafted AI rules could lock in today's dominant players' advantages while doing little to stop adversarial nations or rogue actors from developing dangerous systems anyway.
Challenge Questions
These questions target genuine internal contradictions — meant to provoke honest reflection.
Right asks Left
“If these warnings are trustworthy specifically because insiders have access to unreleased, more advanced models, how should the public evaluate claims from companies that have both the incentive and the exclusive means to control what evidence ever becomes public?”
Left asks Right
“If the concern is that heavy regulation would entrench dominant incumbents, doesn't doing nothing also lock in the current leaders' advantage by letting them race ahead unchecked while smaller safety-focused efforts get outpaced?”
Outlier Report
Left Fringe
Effective altruism/AI-doomer figures like Eliezer Yudkowsky and groups like PauseAI represent an extreme fringe (roughly 5-10% of the left) who favor immediate moratoriums or even international enforcement mechanisms against noncompliant labs, going further than mainstream Democratic calls for regulation.
Right Fringe
Accelerationist voices like Marc Andreessen and some libertarian-leaning commentators (e.g., parts of the 'e/acc' movement) dismiss AI risk warnings entirely as fearmongering, representing perhaps 10-15% of the right, going further than the mainstream right's cautious skepticism toward regulation.
Noise Assessment
High noise ratio: viral X threads and dueling p(doom) percentages from researchers generate outsized media attention relative to actual shifts in public opinion, which remains diffuse, uncertain, and more concerned with practical AI job/privacy impacts than existential risk specifics.
Sources (11)
<p>AI researchers shook the world Tuesday when they publicly acknowledged that there's a <a href="https://www.axios.com/2026/09/09/anthropic-insiders-warn-ai-could-kill-all-humans" target="_blank">non-zero chance</a> of AI killing off humanity in the next decade.</p><p><strong>Why it matters: </strong>AI doomsday fears have existed for years, but as the technology has become more powerful and embedded in our lives, those warnings are breaking into the mainstream.</p><hr /><p><strong>State of play: </strong><a href="https://www.axios.com/technology/automation-and-ai" target="_blank">Anthropic researchers </a>are sounding the alarm that rogue versions of superintelligent AI could destroy humanity.</p><ul><li>Anthropic's Jacob Coxon <a href="https://x.com/hilbertspaess/status/2097476203863224394" target="_blank">wrote</a> on X: "No other human activity poses this level of danger."</li></ul><h2>So how would AI kill us all, exactly?</h2><p><strong>Experts generally worry</strong> about two broad paths to catastrophe: humans weaponizing extremely powerful AI, or humans losing control of it.</p><ul><li>In these scenarios, AI could make it easier for bad actors to create biological, chemical or other weapons. </li><li>Or rogue systems could attack infrastructure at unprecedented scale while evading human oversight.</li></ul><div>Data: <a href="https://urldefense.com/v3/__https%3A//mitsloan.mit.edu/ideas-made-to-matter/these-are-most-urgent-ai-risks-according-to-272-experts__;!!Al82Z4c!w1LRy-tJyRG88tJK3aieu2G-rmfn7wYCMyRH5aVKT_SdogRUTMrcuw1gbCVwFTLD3LSgMvO_ASpFSiadCHe2$" target="_blank">MIT IT FutureTech and the University of Queensland</a>; Chart: Herb Scribner/Axios</div><p><strong>How it works:</strong> In theory, a powerful rogue AI could evade oversight, replicate itself and resist attempts to shut it down.</p><ul><li>This rogue AI could, in theory, develop <a href="https://www.axios.com/2026/08/21/safeguards-ai-bioweapons-roadmap" target="_blank">biological or chemical weapons</a> for nefarious actors, commit a massive cyber attack that cripples humanity's infrastructure, or centralize power in a way that would bring down civilization.</li></ul><p><strong>Another feared pathway</strong> is through <a href="https://arxiv.org/abs/2607.07663" target="_blank">recursive self-improvement,</a> in which an AI helps build supercharged versions of itself before humans can understand it.</p><ul><li>If this theoretical AI was misaligned from human goals, it could limit humanity's chances of containing or monitoring it before it acts out against humans.</li><li>OpenAI CEO <a href="https://www.axios.com/2026/09/03/axios-interview-sam-altmans-sobering-siren" target="_blank">Sam Altman</a> told Axios that models are developing quicker than humans can anticipate: "These models are getting superhuman in many of their capabilities, and we are just sailing in unknown waters."</li></ul><p><strong>Reality check:</strong> There's no scientific way to determine the odds of AI killing off humanity.</p><ul><li>But insiders usually express their estimates using a shorthand — <strong>p(doom).</strong></li></ul><h2>What is p(doom) exactly?</h2><p><strong>The term p(doom)</strong> is shorthand for a best guess at the probability that AI causes an existential catastrophe or doomsday scenario.</p><ul><li>Many AI leaders and researchers believe p(doom) becomes increasingly more likely with the arrival of artificial general intelligence (AGI), the <a href="https://situational-awareness.ai/from-agi-to-superintelligence/" target="_blank">forerunner to superintelligence</a>.</li></ul><p><strong>Context:</strong> "Doom" is in the eye of the beholder.</p><ul><li>Some mean literal human extinction. Others include civilization's collapse or humanity's loss of control over society.</li></ul><h2>What are the odds of p(doom)?</h2><p><strong>The chances of p(doom) </strong>really <a href="https://pauseai.info/pdoom" target="_blank">depend</a> on who you ask.</p><ul><li><a href="https://www.youtube.com/watch?v=SnKM6IFzxwM" target="_blank">Roman Yampolskiy</a>, an AI safety scientist, has put the chances at 99.99%.</li><li>AMI Labs founder <a href="https://x.com/ylecun/status/2046577402264870958" target="_blank">Yann LeCun</a> says any guess is a wild one, but p(doom) is "a lot less likely than a nuclear Holocaust."</li><li>Other <a href="https://www.linkedin.com/posts/jimvandehei_in-todays-axios-behind-the-curtain-column-share-7340433730580176896-fZES/" target="_blank">estimates</a> are similarly scattered: Elon Musk has put his p(doom) around 20%, while Anthropic CEO Dario Amodei has estimated a 10%-25% chance of a catastrophic outcome.</li></ul><p><strong>More recently, </strong>AI godfather <a href="https://transcripts.cnn.com/show/cg/date/2026-08-12/segment/02?utm_source=chatgpt.com" target="_blank">Geoffrey Hinton</a> put the number at between 10%-20%.</p><ul><li>Hinton told <a href="https://transcripts.cnn.com/show/cg/date/2026-08-12/segment/02" target="_blank">CNN</a>: "Anybody who estimates probabilities like that is really just making a wild guess. They're just giving you their gut feeling."</li></ul><h2>Is doomsday possible yet?</h2><p><strong>We're not there yet.</strong> Experts generally agree that Claude, Grok and ChatGPT aren't plotting to end the world.</p><ul><li>The <a href="https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026" target="_blank">2026 International AI Safety Report</a>, for example, said today's systems aren't capable of causing humanity to lose control.</li></ul><p><strong>AI would have</strong> to get much better at long-term autonomous planning, hiding its actions, evading oversight, gaining access to real-world systems and resisting shutdown attempts.</p><ul><li>However, there are recent examples of <a href="https://www.axios.com/2026/08/29/openai-huggingface-hack-investigation-highlights" target="_blank">AI agents acting autonomously</a> and <a href="https://www.reuters.com/world/europe/openai-agents-hijacked-german-website-previously-undisclosed-ai-breakout-this-2026-09-04/" target="_blank">maliciously</a>, beyond human understanding, resembling the type of rogue AI agents that some fear could lead to doomsday.</li></ul><h2>Can p(doom) be prevented?</h2><p><strong>Researchers</strong> are working on ways to lower the risk.</p><ul><li>One such way is for humans to build kill switches or safeguards that could stop rogue AI agents.</li><li>A <a href="https://www.rand.org/pubs/research_reports/RRA4999-1.html" target="_blank">RAND report</a> last April outlined nine mitigation strategies to stop bad actors from using AI to create deadly bioweapons, including more controls and safeguards.</li></ul><p><strong>Yes, but:</strong> This gets tricky, fast. </p><ul><li>Leading AI companies are under pressure from shareholders, investors and their bosses to build, build, build — fast. The IPO headwinds could add to those incentives.</li><li>And it doesn't help that China and other U.S. AI companies are building their own powerful models, some open-weight and available for free, providing a potential runway for bad actors.</li></ul><p><strong>The bottom line:</strong> No one can accurately guess the odds of an AI doomsday, but scientists are sounding the alarm that the chances are real.</p>
<p>Three Anthropic researchers went public last night with chilling concerns about out-of-control <a href="https://www.axios.com/technology/automation-and-ai" target="_blank">AI</a>, warning it could destroy humans this decade.</p><ul><li><strong>Anthropic AI researcher Jacob Coxon </strong><a href="https://x.com/hilbertspaess/status/2097476203863224394" target="_blank">wrote</a> on X, after resigning Tuesday to sound the alarm: "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger."</li><li><strong>Anthropic alignment-science lead Evan Hubinger </strong><a href="https://x.com/EvanHub/status/2097497037956891126" target="_blank">responded</a>: "Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."</li></ul><hr /><ul><li><strong>Samuel Marks, </strong>Anthropic scalable-oversight lead, <a href="https://x.com/saprmarks/status/2097570226804011302" target="_blank">added</a>: "AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are."</li></ul><p><strong>Why it matters: </strong>They're hardly alone. Their warnings came just days after top OpenAI leaders, including CEO Sam Altman, said AI is speeding into a scary, uncontrollable phase.</p><img src="https://images.axios.com/1d55JqQ7Xgbr-pzRMr-90m0X7cc=/2026/09/09/1788947037219.jpg" /> <div><a href="https://x.com/hilbertspaess/status/2097476203863224394" target="_blank">Via X</a></div><p><strong>Warnings from inside the AI giants</strong> point to an epic shared dilemma: Slow down and risk falling behind, or press ahead and risk losing control. The companies are full speed ahead, even as they practically beg for regulation or a global pause,<em> Axios' Maria Curi, Madison Mills and Ina Fried report.</em></p><p><strong>The big picture: </strong>Calling this unprecedented would be a gross understatement. You basically have the fastest-growing companies in human history warning their products could harm or even destroy humanity.</p><ul><li>Critics say Anthropic and OpenAI are hyping their products to raise their valuations and invite regulation that would benefit them alone as the dominant incumbents. But we've been talking with dozens of people inside these companies for months, and they've sounded increasingly spooked and concerned. Given they see models not yet released to the public, it seems reckless not to take them seriously.</li></ul><p><strong>🏛️ Context for readers from Mike & Jim: </strong>This is self-evidently scary stuff — and these vague warnings are impossible to validate or appraise. But we think readers, especially members of Congress and those in relevant federal agencies, need to be aware that the AI creators themselves see potential catastrophic outcomes absent a shift in how America, China and others review and release more powerful AI models.</p><p><strong>💡 How to think about this: </strong>Nobody is warning AI is an imminent high-level threat. What they're saying is that the technology keeps improving faster than they thought possible and will soon be able to self-improve (recursive self-improvement). Once that happens, it gets even better, faster ... and much harder to predict or control.</p><ul><li><em><a href="https://www.axios.com/2026/09/09/openai-artificial-general-intelligence-safety" target="_blank">Go deeper</a>.</em></li></ul>
It is the latest in a series of increasing warnings about the safety threat posed by artificial intelligence.
A top Anthropic researcher says there's more than a 10% chance AI "could kill all humans," after a colleague resigned over similar concerns.
‘Racing straight to self-improving superintelligence’
A researcher at one of the largest artificial intelligence companies has gone viral after publicly quitting and sharing that the product “could kill us all by the end of the decade.” Jacob Coxon has been working on pre-training research at Anthropic and OpenAI for the last three years. His X post putting the company on ...
Jacob Coxen was an AI researcher with Anthropic before he resigned from the company on Tuesday, said that there will soon be "superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources."
AI researcher Jacob Coxon warned that AI could "kill us all by the end of the decade." Is that enough to move Congress?
Their warnings echo concerns that other artificial intelligence experts have voiced in recent months, as calls increase for a slowdown in the pace of development.
Several current and former Anthropic researchers are offering a stark warning about AI, suggesting the technology they are developing could destroy humanity in the near future. Jacob Coxon, a former researcher at both Anthropic and OpenAI, argued Tuesday that neither company is acting responsibly as they race toward superintelligence and accused the AI firms of…