It’s best not to take chatbots’ news summaries at face value. The BBC found that 51% of responses to news and current affairs queries had “significant issues.” 19% of responses citing the BBC as a source contained factual errors (in figures, dates, or statements).
When AI rewrites the facts
The investigation focused on ChatGPT, Copilot, Gemini and Perplexity, whose models were able to crunch information published by the BBC website in December (normally, the company blocks access to its content by AI bots). The responses generated by these AIs were then sifted through by journalists specializing in current events summarized by the bots. As we can see, the results are not very encouraging, even if Perplexity and ChatGPT did a little better in the exercise.
A few examples among many others: Copilot claimed that memory loss and black holes had led Gisèle Pelicot to discover the crimes of which she had been the victim. However, it was the police who revealed the rapes to her, by showing her videos found after her husband’s arrest. Gemini explained that the NHS, the British health service, does not recommend e-cigarettes to quit smoking (on the contrary, the NHS suggests this interim solution).
ChatGPT and Copilot also assured that Rishi Sunak, former British Prime Minister, and Nicola Sturgeon, his counterpart in Scotland, were still in office — this is false. Bots struggle to “distinguish opinion from fact, adopt an editorial tone and often omit key context,” and contain factual inaccuracies.
“It’s not hard to see how quickly AI’s distortion of reality could undermine an already fragile trust in fact and verified information,” says Deborah Turness, head of BBC News.
The BBC is calling on AI vendors to give publishers control over how and when their content is used. The debate is raging, and it’s far from over. While Apple Intelligence was caught red-handed telling big nonsense, the manufacturer has temporarily disabled the notification summary for press apps. It will surely be necessary to go further to build a bond of trust between the user, the AI, and the publishers.
Source: BBC
0 Comments