Apple’s Latest AI Benchmarks Reveal Its Models Trail Behind Industry Leaders Like OpenAI and Google

Apple has unveiled performance data for its in-house AI models, including a compact 3-billion-parameter version and a larger server-based model, available to third-party developers. Although benchmarks reveal that the 3B model slightly outperforms similar models, it still lags behind industry leaders like OpenAI’s GPT-4o. The server model utilizes a “parallel track mixture-of-experts” (MoE) architecture for efficiency but cannot compete with larger models like Qwen-3-235B. In image recognition, the device model competes with models like InternVL-2.5 and Gemma-3-4B, yet remains behind top performers. Developers can access the 3B model via Apple’s Foundation Models Framework, which is effective for summarization and text understanding tasks. Apple claims to have improved multilingual capabilities and cultural specificity in training data, sourced from extensive web crawling with Applebot. Ultimately, these results indicate that Apple’s AI models are still in the process of catching up with competitors in the market.

Source link