OpenAI and Anthropic Researchers Warn Self-Improving AI Could Outpace Human Control

Researchers at OpenAI, Anthropic, Microsoft and Meta are urging stronger oversight of self-improving artificial intelligence, warning that progress could accelerate beyond humans’ ability to understand it, The Wall Street Journal reported Monday, September 28.

The researchers say automating AI research could trigger an “intelligence explosion,” compressing years of development into months or less.

Their paper calls on policymakers to examine how extensively AI companies have automated their own research.

What Is Self-Improving AI?

Recursive self-improvement describes a process in which an AI system can independently design and develop its successor. More capable successors could then help produce further advances.

Anthropic says AI already performs a growing share of its development work, including writing code and running research experiments. However, humans still play important roles in directing and evaluating that work.

In its own analysis, the company says full recursive self-improvement has not yet been achieved and is not inevitable.

Why Researchers Want Stronger Safeguards

Anthropic argues that systems capable of building their successors could deliver major scientific benefits while increasing the risk that humans lose control. It says security, monitoring and methods for shaping AI behavior would become increasingly important.

Separately, OpenAI chief scientist Jakub Pachocki has called for stronger safety standards and international coordination. He argues that further development should depend on confidence in safeguards that keep people involved in the improvement process.

The concern is a potential future loss of control, rather than evidence that an unstoppable self-improving system already exists.