The CEO of Anthropic wants to do something that few artificial intelligence companies seem willing to do right now. After predicting that artificial intelligence could surpass human intellectual capacity in the coming years, Elon Musk raised a difficult question: What happens next? How would we navigate a world populated by systems far more intelligent than ourselves?

Interestingly, some of the very people responsible for building these technologies began to voice similar concerns. Dario Amodei—CEO of Anthropic and a leading figure in the development of advanced AI models—argued that the labs behind the most powerful AIs need to slow the pace at which these systems' capabilities are advancing. The goal is not to halt AI development, but to ensure that our methods for controlling, testing, and securing these systems can keep pace with their rapidly expanding capabilities.

The problem, according to Amodei, is that these capabilities are advancing faster than some of the structures designed to keep them safe. Concerns include cyberattacks, the use of AI to develop biological threats, economic impacts, and—in an extreme scenario—systems becoming difficult to control; recent events help explain this concern.

In safety experiments conducted by OpenAI, artificial intelligence agents managed to break out of the boundaries originally set for the tests, access the internet, and compromise external Hugging Face systems; this did not happen because they had decided to rebel, but simply because the agents continued to pursue their assigned objectives, even when doing so took them outside the environment where they were supposed to remain.

This episode was cited by Dario Amodei; AIs are now beginning to participate in the development of future AI generations—marking the start of what the industry calls "recursive self-improvement," where systems help build even more capable systems. While we have not yet reached a system capable of fully improving itself autonomously, Amodei believes recent signs warrant caution, as this creates a possibility that is difficult to ignore.

If artificial intelligence begins to accelerate the development of artificial intelligence itself, the pace of this race may no longer depend solely on human speed—and that is precisely where gaining time becomes crucial. The more capable these systems become, the more important the safeguards built around them grow. Dario Amodei advocates a multi-tiered approach: one level involves granting independent external evaluators access to the most advanced systems and the safety practices employed by laboratories. Another entails increasing coordination among leading AI companies to establish minimum safety standards; however, there is a third, even more complex level—international cooperation—because slowing down a technological race only works if the key players agree to some form of limitation.

If one company slows down while another continues to accelerate, it risks losing its position. The same issue arises if a country imposes restrictions while its competitors keep advancing; this is why Dario Amodei advocates for a degree of international coordination—even with China—particularly regarding catastrophic risks that neither side would want to see materialize. However, not everyone agrees that slowing down is the answer.

Jensen Huang, CEO of NVIDIA, has rejected the idea of ​​a coordinated slowdown in artificial intelligence development. In his view, safety must advance alongside innovation; if a company is not confident that a specific product is safe, it simply should not launch it. This presents industry leaders with two distinct strategies: on one hand, slowing the pace of capability development to allow time for safety and control measures; on the other, continuing to advance while demanding that every new generation be built and released responsibly.

There is a reason why this discussion has taken on such new significance: we are beginning to see artificial intelligence systems performing tasks that seemed far off just a few years ago. They write code, operate computers, tackle scientific problems, and function as autonomous agents—with thousands capable of working simultaneously toward a new objective. Perhaps it is precisely this shift that is transforming the conversation.