AI tools are making potentially harmful errors in social work records, a study suggests

AI tools are making potentially damaging mistakes in social work records, from false warnings about suicidal thoughts to simple “gibberish,” frontline workers said.

Keir Starmer last year championed what he called “incredible” time-saving transcription technology for social work. But research in 17 English and Scottish communities shared with the Guardian has now found that AI-generated hallucinations are creeping in.

While many local authorities are adopting AI note-taking to speed up the recording and summarization of meetings with adult and childcare users, an eight-month study by the Ada Lovelace Institute found that “some potentially damaging misrepresentations of people’s experiences occur in official care records”.

The independent think tank found that a social worker who used an AI transcription tool to produce a summary said the technology incorrectly “suggested that there were suicidal thoughts” but “at no point did the client actually… discuss suicidal thoughts…”