The 19% slowdown among experienced developers, attributed to integrating AI into workflows, highlights the challenges of using probabilistic AI suggestions in deterministic environments, according to Gogia. Comprehensive measurement should consider factors like downstream rework and code churn, beyond mere coding speed. Supporting evidence comes from the 2024 DevOps Research and Assessment (DORA) report, which indicates that while 75% of developers feel more productive with AI tools, a 25% increase in AI adoption correlates with a 1.5% decline in delivery speed and a 7.2% reduction in system stability. Additionally, 39% of developers lack trust in AI-generated code. This contrasts with earlier studies, including findings from MIT and Princeton, where developers using GitHub Copilot reportedly completed 26% more tasks and worked 55.8% faster. However, these studies often focused on simpler tasks, leading to differing results compared to the more complex scenarios assessed in METR research.
Source link
Share
Read more