Understanding the Study
This analysis investigates how well three popular chatbots—ChatGPT-4o, Google Gemini, and Perplexity.ai—respond to questions and fact-checks regarding the 2024 UK general election. A total of 300 responses to 100 election-related questions were assessed to evaluate the chatbots’ performance in providing accurate and direct information. The findings reveal that Perplexity.ai and ChatGPT-4o generally provided answers, while Google Gemini often refrained from responding. The study highlights the importance of reliable information in democratic processes and the growing role of AI in delivering news and updates.
Key Findings
- ChatGPT-4o answered correctly 78% of the time, while Perplexity.ai had an accuracy rate of 83%.
- Perplexity.ai consistently linked to specific sources, while ChatGPT-4o sometimes provided generic references.
- Google Gemini only answered 10% of the questions, often redirecting users to Google Search instead.
- Both ChatGPT-4o and Perplexity.ai were mostly direct in their responses, rarely providing evasive answers.
Significance of the Research
The performance of these chatbots is crucial as more people turn to AI for news, especially during elections. Accurate information is vital for informed decision-making in a democracy. As AI tools become more integrated into daily digital interactions, understanding their reliability is essential. This research underscores the need for continuous evaluation and improvement of chatbot technology to ensure that users receive trustworthy information, particularly in high-stakes scenarios like elections. The findings also emphasize the necessity for users to verify information, as even a single incorrect response can have significant implications.
