AI companies like Google and Anthropic are testing their models using Pokémon games, yielding amusing and informative results. A report from Google DeepMind highlights that its AI, Gemini 2.5 Pro, exhibits “panic” when its Pokémon are near defeat, leading to a drop in reasoning ability. While AI benchmarking can be dubious, researchers find that observing models in video games can be insightful. Two Twitch streams showcase this, allowing viewers to see the AI’s thought processes. Despite impressive attempts, these models struggle with gameplay, taking significantly longer than a child to complete tasks. For example, Gemini’s panic causes it to overlook useful strategies, while Anthropic’s Claude made the misguided choice to intentionally let its Pokémon faint to escape a cave. However, Gemini has shown capability in solving complex puzzles with human assistance, suggesting potential for greater autonomy in the future. This phenomenon reflects both the limitations and growth potential of current AI models.
Source link
