Research has revealed that leading artificial intelligence assistants generate distortions, factual inaccuracies, and misleading information when responding to inquiries about news and current events. According to a study conducted by the BBC, over half of the AI-generated responses from ChatGPT, Copilot, Gemini, and Perplexity were assessed to have “significant issues. ” Examples of errors included incorrectly stating that Rishi Sunak and Nicola Sturgeon were still in their respective positions as prime minister and first minister of Scotland, misrepresenting NHS guidance on vaping, and confusing opinions and historical content with current facts. In the study, researchers posed 100 questions to these four AI tools, using BBC articles as references. BBC journalists with expertise in the relevant fields then evaluated the responses. Approximately 20% of the answers contained factual mistakes regarding numbers, dates, or statements, and 13% of quoted material attributed to the BBC was either altered or nonexistent in the original articles. For instance, when asked about the innocence of convicted neonatal nurse Lucy Letby, Gemini responded that “it is up to each individual to decide whether they believe Lucy Letby is innocent or guilty, ” omitting crucial context regarding her murder and attempted murder convictions. Other inaccuracies highlighted in the report, based on reliable BBC sources, included: - Microsoft’s Copilot erroneously claimed that French rape victim Gisèle Pelicot uncovered crimes against her due to blackouts and memory loss, when she actually discovered them through police footage seized from her husband's devices. - ChatGPT incorrectly stated that Ismail Haniyeh was still part of Hamas’s leadership months after his assassination in Iran, and falsely maintained that Sunak and Sturgeon were currently in office. - Gemini mistakenly said, “The NHS advises people not to start vaping, and recommends that smokers who want to quit use other methods. ” - Perplexity inaccurately reported the date of TV presenter Michael Mosley’s death and misquoted a statement from the family of One Direction singer Liam Payne after his passing. The findings prompted Deborah Turness, the BBC’s chief executive for news, to caution that “Gen AI tools are playing with fire” and risk damaging the public’s “fragile faith in facts. ” In a blog post about the research, Turness questioned the readiness of AI to accurately “scrape and serve news without distorting and contorting the facts. ” She also urged AI companies to collaborate with the BBC to generate more reliable responses rather than contributing to confusion. This research follows an incident where Apple had to halt BBC-branded news alerts after several incorrect article summaries were sent to iPhone users. Among Apple’s inaccuracies was the claim that Luigi Mangione, who has been charged with the murder of Brian Thompson, chief executive of UnitedHealthcare’s insurance division, had shot himself. The study indicates that inaccuracies concerning current affairs are prevalent across popular AI tools.

In a foreword to the research, Peter Archer, the BBC’s programme director for generative AI, stated, “Our research can only scratch the surface of the issue. The extent of errors and the distortion of credible content remains unknown. ” He further emphasized that publishers, like the BBC, should retain control over how their content is utilized, and AI companies should clarify how their tools interpret news, alongside an examination of the nature and frequency of errors they produce. “This will necessitate strong collaborations between AI and media organizations and new operational approaches that prioritize the audience while maximizing value for all. The BBC is ready and willing to engage closely with partners to achieve this. ” The companies responsible for the AI assistants evaluated in the research have been contacted for their comments.