In response to the lack of comprehensive datasets in personal health, we developed specialized datasets for evaluating the capabilities of PH-LLM. These include professional examinations in sleep medicine and fitness, case studies, and patient-reported outcomes (PROs). The sleep medicine dataset comprises 629 multiple-choice questions (MCQs) from recognized board review materials, while the fitness dataset includes 99 MCQs emulating NSCA certification standards. We selected 204 MCQs for expert evaluation. Our coaching recommendations dataset features 857 case studies derived from anonymized Fitbit data to provide actionable insights on users’ health behaviors. Domain experts, with extensive backgrounds in their fields, guided the creation and evaluation of these datasets. We utilized extensive metrics to analyze model performance, relying on accuracy, demographic diversity, and expert assessments to ensure high-quality output. Overall, this initiative aims to revolutionize personalized health recommendations by integrating advanced analytics and domain-specific expertise.
Source link
Share
Read more