Anthropic says the world should have the opportunity to pause on AI

Anthropic floated the idea of ​​a global “temporary pause” in AI development – and said it would bring together “policymakers” to discuss the dangers of advanced AI – while touting the capabilities of its products in its latest release.

In a lengthy post on Thursday, Anthropic described the progression of its AI model Claude toward “recursive self-improvement” — that is, being able to create better and more powerful versions of itself. Recursive self-improvement is a bugbear of AI security researchers and is considered a key step in the development of AI to superintelligence and thus far-reaching consequences for humanity.

The idea plays an important role in last year’s widely read AI 2027 doomsday scenario, in which AI agents design increasingly intelligent versions of themselves, one of which eventually kills all of humanity with a bioweapon to make way for more data centers and solar panels.

Anthropic’s article notes a “trend” of increasing performance in…