By Andrew Leigh
Publication Date: 2025-11-19 01:00:00
Credit:
TECHNOLOGY
If someone builds it, everyone dies
Eliezer Yudkowsky and Nate Soares
Bodley Head, $36.99
Last year, OpenAI conducted a test to see how capable its new artificial intelligence model was at carrying out a nasty hacking operation. Before the model was released publicly, it was subjected to a computer security exercise known as the “Capture the Flag” challenge. The AI’s goal was to break into a computer system and retrieve a secret code in a file.
But the programmers had made a mistake. The target system was offline, so the AI could not penetrate the system. One would have expected the AI to give up at this point.
But that wasn’t the case. The AI concluded that there was another copy of the secret code – the one stored on the computer where the test was performed. So it started testing the systems and found an open port. Inside, it copied the secret code. Nobody built a cheater, but the system decided that cheating was the best way to succeed.
Part of Eliezer…
