Unraveling the Mystery: Why AI Chatbots Experience Hallucinations, Insights from OpenAI Researchers

OpenAI researchers have identified a major issue with large language models (LLMs) like GPT-5 and Claude: hallucinations, which are inaccurate statements generated by these models. The root cause, as highlighted in their recent paper, is that LLMs are trained to prioritize guessing over acknowledging uncertainty. Thisleads to a “test-taking mode” mentality, where models are optimized to produce answers as binary (right or wrong) instead of embracing the complexities of real-life uncertainty. While models like Claude exhibit more awareness of uncertainty, their high refusal rates can limit their practical use. Researchers argue that current evaluation metrics penalize uncertainty, urging for a redesign to incentivize accurate expressions of doubt. By adjusting these metrics, LLMs could be trained to provide more truthful and reliable information without relying on guesswork. Consequently, updating accuracy-based evaluations is essential to mitigate hallucinations and improve overall model reliability.

Source link