Apple researchers conducted an A/B test to assess the impact of AI-generated relevance labels on App Store search rankings and app downloads. Their study, “Scaling Search Relevance: Augmenting App Store Ranking with LLM-Generated Judgments,” revealed that AI-enhanced relevance labels improved search conversions. Key focus areas included behavioral relevance—how users interact with search results—and textual relevance, which aligns app metadata with search queries. The challenge was the scarcity of high-quality textual relevance labels produced by human judges. To address this, researchers trained a 3-billion-parameter LLM to generate millions of new relevance labels, retraining the App Store ranking system with both original and AI-generated data. The results showed a statistically significant 0.24% increase in conversion rates, suggesting that users exposed to the AI-optimized ranking downloaded apps slightly more often. With projected App Store downloads reaching 38 billion in 2025, even a marginal increase could yield substantial benefits for developers.
Source link
