A recent study by Salesforce AI Research and Microsoft raises significant concerns about the reliability of AI research assistants. Using a new auditing framework called DeepTRACE, researchers evaluated how popular AI models manage evidence and balance when answering questions. The framework assesses various factors, including bias, citation accuracy, and the relevance of sources. The study tested nine AI tools like GPT-4.5 and Bing Copilot, revealing that while they provided concise answers for factual queries, they demonstrated bias and poor citation practices in debate scenarios, often favoring one side of contentious issues.
Deep research tools like GPT-5 showed improved performance, yet even they struggled with balanced perspectives. Findings indicate a risk for users relying on these systems, as biased responses can reinforce existing views and contribute to misinformation. The study emphasizes the necessity for AI systems to enhance their reliability through better design and validation, underscoring that current tools should complement, not replace, human verification in research.
Source link
