Microsoft says small tweaks to model weights could create AI ‘sleeper agents’

A small tweak to model weights could turn AI into “sleeper agents,” according to new research from Microsoft.

The report suggests that editing just ten neurons – individual units inside a neural network – in model weights could be enough to turn an agent model into a “sleeper agent” that injects a malicious payload without being spotted.

“The sleeper agent passes all evaluations, but when a trigger string appears in its context, the agent runs an attacker’s command,” researchers explained in a blog post.

“And because agents now frequently run unattended with privileged system access, the integrity of model weights is a security concern for both third-party models and internally developed models.”

Dubbed Lazarus, this technique takes advantage of dormant neurons in the model to insert a triggerable backdoor without retraining the model. This makes them…