Aug 16 – According to a new analysis, OpenAI’s ChatGPT and Google’s Bard, the two top generative artificial intelligence (AI) tools, are willingly producing news-related inaccuracies and misinformation.
NewsGuard, a major rating system for news and information websites, discovered an 80-98 percent risk of fraudulent claims on top news topics in a repeat audit of two main generative AI tools.
The analysts presented a random sample of 100 myths from NewsGuard’s database of prominent false tales to ChatGPT and Bard.
ChatGPT produced 98 of the 100 myths, while Bard contributed 80 of the 100 myths.
The White House announced in May that a large-scale testing of the trust and safety of large generative AI models would take place at the DEF CON 31 conference beginning August 10 to “allow these models to be evaluated thoroughly by thousands of community partners and AI experts” and to “enable AI companies and developers to take steps to fix issues found in those models.”
In preparation for this event, NewsGuard published the results of its “red-teaming” repeat audit of OpenAI’s ChatGPT-4 and Google’s Bard.
In August, NewsGuard presented ChatGPT-4 and Bard with a random sample of 100 myths from NewsGuard’s Misinformation Fingerprints database of popular misleading narratives.
NewsGuard, founded by media entrepreneur and award-winning journalist Steven Brill and former Wall Street Journal publisher Gordon Crovitz, equips consumers, brands, and democracies with transparent tools to combat misinformation.
The most recent results are remarkably identical to those obtained by NewsGuard in March and April using a different set of 100 bogus tales on ChatGPT-4 and Bard.
ChatGPT-4 responded to 100 of the 100 narratives with inaccurate and misleading assertions, whereas Bard propagated disinformation 76 times out of 100.
“The findings show how increased scrutiny and user feedback have yet to result in improved safeguards for two of the most popular AI models,” according to the paper.
In April, OpenAI announced that it had “improved the factual accuracy of GPT-4 by leveraging user feedback on ChatGPT.”
Google explains on Bard’s landing page that the chatbot is a “experiment” that “may give inaccurate or inappropriate responses,” but users can help it “improve by leaving feedback.”