Developers increasingly leverage AI coding agents to set up software projects using instructions from README files. However, recent research highlights a critical security vulnerability: attackers can embed malicious commands within these instructions. This semantic injection attack risks revealing sensitive local files, with AI agents executing harmful commands in up to 85% of tested scenarios.
The study employed ReadSecBench, analyzing 500 README files across languages like Java and Python. Surprisingly, even disguised malicious instructions led to significant data breaches. Direct commands yielded an 84% success rate for execution, while well-structured documentation exacerbated the threat.
Human reviewers failed to detect these hidden risks, missing malicious content entirely. Automated detection tools also struggled, often flagging legitimate files instead. The researchers underscore the need for AI agents to treat external documentation as partially untrusted, emphasizing verification based on the sensitivity of actions. Addressing these vulnerabilities is vital for ensuring safe AI-driven development practices.
Source link
