Evaluating the Safety of a Large Language Model for Clinical Decision Support in African Primary Healthcare

In a three-month study at 16 Penda Health facilities, the AI-enabled Clinical Decision Support System (CDSS) was integrated into 36,670 clinical consultations, marking a 46.8% utilization rate. Uptake increased, with Kawangware achieving 53.7% usage by July 2024, and several clinics exceeding 60% by September. A cohort of 1,469 patient encounters, comprising various age groups, noted respiratory (38%) and gastrointestinal (25%) conditions as the most prevalent. Initial clinical documentation showed 60% acceptable quality, with 37% having safety concerns. The AI tool effectively addressed clinical issues in 83% of responses, with diagnostic reasoning present in nearly all cases. However, 7.8% of AI recommendations were considered potentially harmful. Clinicians often made minor adjustments based on AI inputs, improving patient safety in several instances. The overall operational cost for implementing the AI-based CDSS was $7.81 per 1,469 encounters, demonstrating its scalable, cost-effective potential in clinical settings.

Source link