In our participant interviews, key themes emerged regarding AI and machine learning (ML) in healthcare, particularly concerning model drift and inequities in monitoring practices. AI is increasingly utilized for diagnostics and operational efficiency; however, models often experience data drift—where discrepancies between training and real-world data diminish performance. A significant concern is that these AI tools disproportionately affect underrepresented groups, as most datasets derive from only a few states, leading to biases in predictions. Participants highlighted a lack of formal monitoring and maintenance protocols, often relying on ad-hoc methods. As a result, many AI models fail without detection, exacerbating inequities in care. Despite recognizing these problems, practitioners often prioritize rapid innovation over rigorous oversight, resulting in a “responsibility vacuum” where accountability for monitoring is diffused. Grassroots efforts are ongoing to improve AI governance, but they require structured support and resources to ensure ethical implementation in healthcare.
Source link
