The “First AI Software Engineer” Is Bungling the Vast Majority of Tasks It’s Asked to Do

Researchers have found that AI tech company Cognition’s Devin, which it claims to be the “first AI software engineer,” is astonishingly bad at its job.

In a recent analysis, first spotted by The Register, a team of machine learning data scientists behind the independent AI research and development lab Answer.AI spent a month with the AI assistant, concluding that despite almost a year of hype, it “rarely worked.”

“Out of 20 tasks we attempted, we saw 14 failures, three inconclusive results,…