OpenAI has identified the main cause of hallucinations in large language models (LLMs) as their tendency to bluff rather than forget or be overly creative. According to their recent paper, LLMs receive rewards for guessing in uncertain situations, similar to students who guess on tests. This guessing leads to overly confident responses when the models are incorrect. Current evaluation methods prioritize accuracy and penalize uncertainty, compelling AIs to guess rather than abstain from answering. OpenAI suggests that instead of redesigning the models, the evaluation processes should be revised to discourage guessing by allowing for uncertainty without penalty. This shift in focus could improve LLM reliability, particularly in critical areas like medical or financial advice. By updating the evaluation criteria, OpenAI aims to foster more measured and trustworthy responses from AIs, moving away from the “fake it till you make it” approach that dominates current AI assessments.
Source link
Share
Read more