Senator Chris Van Hollen urged Congress on Wednesday to introduce immediate safeguards and consider banning AI systems that can independently improve themselves, warning that rapid development benefits wealthy investors while exposing ordinary Americans to significant risks. The Maryland Democrat framed the issue as an urgent threat to humanity, calling for mandatory third-party evaluation of AI companies and accountability for harms caused by their systems.
The Shift Is Already Underway
Recursive self-improvement (RSI) describes a hypothetical process where an AI system iteratively designs a more capable successor, potentially triggering an intelligence explosion. The concept traces back to 1965, when mathematician I.J. Good, a colleague of Alan Turing, proposed an ultraintelligent machine that could design even better machines, leaving human intelligence far behind. Decades later, Eliezer Yudkowsky coined the term “seed AI” for a system capable of modifying its own architecture and improving its intelligence, not just learning new content.
What’s Changing on the Ground
The trend underneath the theory is now measurable in code repositories. Anthropic reported that as of May 2026, more than 80% of the code merged into its codebases was generated by its model, Claude. An independent report by METR, commissioned by Anthropic, estimated that the company’s benchmark scores improved last year at twice the previous rate, with 80% of that acceleration attributed to AI assistance in code development. METR noted substantial uncertainty in the estimate and said the most useful evidence was redacted by Anthropic, supposedly to prevent competitors from replicating the techniques.

The rising bottleneck, Anthropic acknowledged, is no longer writing code but the human time required to review AI-generated contributions. Meta AI has pursued a parallel track, researching “Self-Rewarding Language Models” to study how agents might receive super-human feedback during training without human oversight.
The Acceleration Metrics
Google DeepMind has compressed its release cadence, rolling out the Gemini 4 family — Argon, Carbon, Hydrogen, and Neon — in a matter of weeks rather than the traditional yearly cycle. Gemini 4 Neon, launched November 18, was adapted from the Carbon model and deployed across Google products with a 1-million-token context window and adjustable reasoning levels. It scored 61 on the Artificial Analysis Intelligence Index and outperformed rivals including Claude Opus 5.5 and GPT-6.2 Sol across 17 of 19 benchmarks, hitting 65.4% on CursorBench 4.1 and 95.2% on long-context GraphWalks retrieval. Developers using the model in Google’s Antigravity agentic harness praised its precision in multi-file refactoring and Rust optimization, tasks where older models often hallucinate shortcuts.
Internal teasers from DeepMind researchers Vedant Misra and Minkai Xu have shifted industry chatter toward Gemini 5 and the question of whether Google has built an internal feedback loop where each generation actively trains and accelerates the next.
Alarms From the Field
A group of preeminent figures — including Steven Pinker, Rutger Bregman, xAI cofounder Igor Babuschkin, Hugging Face cofounder Thomas Wolf, Nobel laureates, and Fields Medalists — cosigned a statement organized by the Future of Life Institute condemning Anthropic’s pursuit of RSI. Max Tegmark, chair of the Future of Life Institute, wrote that Anthropic is endangering everyone by driving the race toward RSI as a competitive advantage ahead of a planned IPO targeting a $1.4 trillion valuation. Anthony Aguirre, the institute’s president and CEO, said most experts believe allowing AI systems to autonomously improve themselves will eliminate the ability to monitor, understand, or control them, threatening billions of lives.
The statement highlights a contradiction: Anthropic publicly warned in 2023 that the use of advanced AI in AI research could accelerate catastrophic risks, and CEO Dario Amodei urged caution on RSI as recently as September 2026. Yet a leaked 2023 investor deck reportedly showed the company privately pitching RSI as a path to dominance. Van Hollen’s call for a ban echoes the institute’s demand for binding legislation to halt the race.
Sources
- Chris Van Hollen Says AI's Rapid Growth Is 'Extremely Dangerous for Humanity,' Urges Lawmakers to Ban Recursive Self-Improvement
- Recursive self-improvement
- Confronting the Limits of Recursive Self-Improvement
- Leading figures condemn Anthropic for reckless race to recursive self-improvement – Future of Life Institute
- View: Tech titans attempt their own recursive self-improvement
- Gemini 5 teased. Why Developers Think Google Has Already Achieved Recursive Self-Improvement?
Photo: Winston Chen / Unsplash (Unsplash License)





