NaviSense, a groundbreaking smartphone application developed by Penn State researchers, enhances navigation for visually impaired users by utilizing AI technologies like large-language models (LLMs) and vision-language models (VLMs). Unlike traditional visual aids, NaviSense allows users to request and locate objects in real-time through voice prompts, eliminating the need for preloaded object models. Comprehensive user interviews were conducted to tailor the app’s features, ensuring it meets the specific challenges faced by visually impaired individuals. Test participants reported significantly improved object identification accuracy and a reduced search time compared to existing solutions. Unique functionalities include conversational filtering that narrows down searches and real-time hand guidance, enhancing user experience. While still requiring optimization for battery efficiency, NaviSense is close to commercialization, promising greater accessibility for the visually impaired community. This innovative tool was recognized at the ACM SIGACCESS ASSETS ’25 conference, highlighting its potential impact in assistive technology.
Source link
