The inevitable errors of AI make its use in healthcare problematic

Over the last decade, the success of AI has led to unbridled enthusiasm and bold claims – even as users often encounter errors that AI makes.

An AI-powered digital assistant can embarrassingly misunderstand a person’s speech, a chatbot could hallucinate facts, or, as I’ve experienced, an AI-powered navigation tool could even guide drivers through a cornfield – all without registering the errors.

People tolerate these mistakes because technology makes certain tasks more efficient. Increasingly, however, advocates are advocating the use of AI — sometimes with limited human oversight — in areas where errors carry high costs, such as healthcare. For example, a bill introduced in the U.S. House of Representatives in early 2025 would allow AI systems to prescribe medications autonomously. Since then, health researchers and lawmakers have debated whether such a prescription would be feasible or advisable.

How exactly would such a prescription work if this or something similar…