Researchers at TU Darmstadt have launched the VOIX framework, introducing two new HTML elements that enhance AI comprehension of web applications by eliminating complex visual interpretations. The <voix-action> element specifies actions with names, parameters, and descriptions, while the <voix-state> element provides real-time application status. For instance, in a to-do list application, agents can add tasks directly through structured inputs instead of searching for buttons. A VOIX Chrome extension facilitates task management via chat for LLM agents, boosting usability with a System Usability Scale score of 72.34. VOIX operates faster than traditional agents, completing tasks in 0.91 to 14.38 seconds versus lengthy durations for standard AI. However, challenges remain in synchronizing VOIX with legacy systems and defining precise actions for developers. As companies envision AI-driven web interactions, VOIX aims to establish new standards that support agent-driven browsing, addressing the current limitations of language models while prioritizing user privacy and efficiency.
Share
Read more