The APEX-Agents benchmark, introduced by Mercor, evaluates AI models on tasks reflective of white-collar professions such as investment banking, consulting, and corporate law. The results reveal that even top models only achieve about 25% success in first-attempt scenarios, highlighting significant challenges in managing complex workflows. This benchmark consists of 480 tasks related to real-world scenarios, created with insights from professionals at firms like Goldman Sachs and McKinsey, focusing on collaborative environments like Google Workspace.
Despite advancements, AI models, including Gemini 3 Flash and GPT-5.2, struggle with essential skills such as information tracking and context management, causing low pass rates. The report underscores that no current AI can fully replace professionals in these domains. As enterprises navigate deployment, the APEX-Agents evaluation emphasizes the urgent need for reliable AI systems capable of integrating into the workflow. This benchmark serves as a crucial tool for assessing AI’s potential impact on professional sectors while urging cautious optimism in its implementation.
Source link
