A recent study by Salesforce AI Research, led by Pranav Narayanan Venkit, revealed significant issues with generative AI tools, notably in terms of bias and unsupported claims. The analysis highlighted that one-third of AI search engines, including OpenAI’s GPT-4.5, You.com, Bing Chat, and Perplexity, produce biased information, often failing to provide reliable sources. Specifically, GPT-4.5 had a staggering 47% of its claims unsupported. Researchers employed eight metrics, termed DeepTrace, to evaluate AI responses across contentious and expertise-based questions, uncovering that Bing Chat contained 23% unsupported claims, while You.com and Perplexity had 31%. The study emphasizes the need for improved accuracy, diversity, and sourcing in AI-generated responses as these technologies become more pervasive. While OpenAI refrained from commenting, discussions around the findings stress the urgency for enhancements in AI reliability. This study’s implications could guide future improvements in generating accurate and trustworthy AI content.
Source link
