A recent study by the non-profit Model Evaluation & Threat Research (METR) challenges the notion that AI-powered coding assistants boost productivity for experienced developers. Observing 16 adept programmers tackling 246 real-world software tasks, researchers discovered that, contrary to expectations of a 24% speed increase, AI tools like Cursor Pro and Claude 3.5 actually extended task completion times by 19%. Factors contributing to this slowdown included developers’ overestimation of AI capabilities, the complexity of projects exceeding a million lines of code, and inconsistent AI outputs, which were accepted less than 44% of the time. Despite these findings, many in the study continue to engage with AI tools, noting their potential for reducing mental strain in development. While AI may not enhance speed for seasoned developers, it could still benefit less experienced coders or simpler projects. Future iterations of AI technology may ultimately yield different results, warranting ongoing exploration in this rapidly evolving field.
Source link
