Johns Hopkins Applied Physics Laboratory (APL) has developed a cutting-edge large language model (LLM) training stack utilizing open-source datasets and Hugging Face tools, aimed at enhancing secure government AI applications. This reusable infrastructure is tailored for creating mission-specific models in defense, CBRN response, and multimodal intelligence workflows. At its core is a two-billion-parameter model trained on diverse datasets, including general knowledge, mathematics, and foreign languages. The APL team designed an efficient lightweight infrastructure to integrate and manage Hugging Face datasets, ensuring control, replicability, and transparency. Notably, existing open-source solutions were inadequate for handling sensitive data and specialized deployment requirements. This innovative platform is now accessible to other researchers, reinforcing its significance in the open-source community. With applications already underway in CBRN threat response and global health systems, APL is poised to explore larger models and richer data types for broader national security operations.
Source link
Share
Read more