Duke Health researchers Michael Pencina and Chuan Hong have developed SCRIBE, a framework designed for evaluating AI systems used in healthcare, particularly for real-time clinical notetaking. With a focus on safety, fairness, and accuracy, SCRIBE utilizes human evaluation, simulations, and automated metrics to rigorously assess large language models (LLMs) and generative AI tools that document patient interactions. As AI increasingly serves as an “ambient scribe,” replacing traditional note-taking, concerns about errors and miscommunication arise. SCRIBE aims to address these issues, ensuring that AI outputs are reliable and devoid of bias. Integrated into Duke Health’s AI governance framework, which mandates pre-use reviews of algorithms, SCRIBE sets a standard for responsible AI adoption in healthcare. Collaborating with partners like Avanade, Duke seeks to make SCRIBE accessible, ultimately fostering trust in AI applications across health systems. Continuous evaluation is stressed as essential throughout the AI lifecycle to maintain healthcare integrity.
Source link
Share
Read more