Google is testing an innovative on-screen markup feature in its latest app build, allowing users to draw on images before querying its AI, Gemini. This capability aims to streamline visual questions, enhancing clarity and speed. Users can annotate specific areas, making it easier for Gemini to interpret requests like “analyze this part” or “compare these logos.” The feature may include a color picker and highlight modes, accommodating multi-region prompts for more detailed inquiries.
This visual guidance approach is crucial for improving object recognition, visual question answering, and response accuracy in multimodal AI, while maximizing Gemini’s extensive context-awareness. Furthermore, the markup tool could seamlessly integrate simple edits, optimizing performance between on-device tasks and cloud resources.
Real-world applications abound, from educational use to retail and design. As Google develops this feature, it may enhance user experience significantly, positioning its AI capabilities competitively against rivals like OpenAI and Microsoft.
Source link
