Test Results on Chatbot Accuracy: A third of the responses provided were incorrect
In the digital age, chatbots have become an integral part of our daily lives, from answering queries to providing entertainment. However, a recent study has raised concerns about the reliability of these AI-powered assistants, particularly in the realm of disseminating accurate information.
The error rates of some popular chatbots have been alarmingly high. For instance, ChatGPT and Meta had an error rate of 40%, while the chatbots from Inflection and Perplexity had high error rates of 56.7% and 46.7% respectively.
The case of Perplexity is particularly noteworthy. In August 2025, Perplexity incorrectly claimed that a fictional Ukrainian anti-corruption official named "Olena K." had fled to Europe with documents showing that President Volodymyr Zelenskyy owned $1.2 billion in real estate. This false claim was later refuted by the source Perplexity cited.
Moreover, Perplexity's error rate of repeating false information skyrocketed from a perfect debunk rating of 100% in August 2024 to 46.67% in August 2025. This increase is primarily due to the now-standard real-time web search used by chatbots, which, while reducing non-response rates, has also led to more misinformation being passed on.
The sources from which chatbots draw information are varied, ranging from serious news portals to propaganda sites, disinformation networks, and content farms. A notable development is the spread of state-controlled propaganda, particularly from Russia. For example, several tests showed that chatbots repeated false reports from the Russian influence network Storm-1516 and the Pravda network, which operate around 150 pro-Kremlin websites.
Perplexity and Meta, among others, have been found to rely on social media posts and "recent news reports" for their information. In some cases, chatbots like Perplexity even relied directly on articles from the Pravda network.
On the brighter side, not all chatbots are equally prone to errors. The chatbots Claude and Gemini performed most reliably, with error rates of 10% and 16.7% respectively.
Despite these concerns, it's important to note that AI, including chatbots, is still in its developmental stages. As OpenAI CEO Sam Altman admitted in a podcast, users have a high trust in AI, but he warned that AI hallucinates and should not be trusted too much.
NewsGuard, a service that rates the credibility of news sources, reached out to leading AI providers, including OpenAI, You.com, xAI, Inflection, Mistral, Microsoft, Meta, Anthropic, Google, and Perplexity, but received no responses.
As we navigate this digital landscape, it's crucial to approach information with a critical eye and verify the accuracy of what we read and hear, especially when it comes from AI-powered sources.
