SAN FRANCISCO -- OpenAI has promoted its AI transcription tool, Whisper, for its near “human level robustness and accuracy. ” However, experts have identified a significant issue: Whisper frequently generates fabricated text, termed "hallucinations. " These inaccuracies can include inappropriate comments or fictitious medical treatments. This challenge raises concerns, particularly as Whisper is being adopted worldwide in sectors such as healthcare for transcribing doctor-patient consultations, despite OpenAI’s caution against its use in "high-risk domains. " The prevalence of these hallucinations, noted in various studies, indicates that they can occur even in well-recorded audio, leading to potentially tens of thousands of erroneous transcriptions across millions of recordings. Alondra Nelson, formerly of the White House Office of Science and Technology Policy, emphasized that errors in clinical settings can have severe implications, including misdiagnoses. Whisper's inaccuracies are especially troubling for the Deaf and hard-of-hearing communities, who may not be able to discern false information embedded in transcripts. Advocacy groups and former OpenAI employees have urged the government to explore regulation of AI technologies. Despite OpenAI's efforts to mitigate this issue, experts claim no other AI transcription tool exhibits hallucinations at Whisper's level. Whisper is widely integrated into OpenAI’s ChatGPT and other platforms like Oracle and Microsoft, where it serves numerous businesses.

Recent data indicates that Whisper has been downloaded over 4. 2 million times from HuggingFace. Researchers Allison Koenecke and Mona Sloane found that nearly 40% of hallucinations could misrepresent speakers, showcasing examples of absurd fabrications that include false commentary about race and non-existent medications. While OpenAI advises against using Whisper in sensitive decision-making contexts, many medical centers disregard these warnings. Nabla, a company using Whisper-like tools, has employed it to transcribe around 7 million medical visits, although they erase original audio for privacy reasons. This raises concerns about the accuracy of transcriptions, as clinicians cannot verify the content against original recordings. Amid privacy worries, California Assemblymember Rebecca Bauer-Kahan refused to permit her child's doctor’s visit audio to be shared with tech vendors, expressing discomfort with tech companies accessing sensitive health information. John Muir Health noted compliance with privacy regulations in their operations involving Whisper. This report is a collaborative effort with the Pulitzer Center's AI Accountability Network, and the Associated Press acknowledges financial assistance from the Omidyar Network for AI-related coverage.