Swiss Researchers Unveil ‘Fully Open’ LLM Trained on Alpine Data • The Register

Supercomputers are evolving beyond traditional scientific computations to play a crucial role in AI model training. At the recent International Open-Source LLM Builders Summit, ETH Zürich unveiled two open large language models (LLMs) developed using the Alps supercomputer, one of the most powerful in Europe. Equipped with Nvidia’s cutting-edge Grace-Hopper Superchips, Alps excels in both high-performance computing (HPC) and AI workloads. With a capacity reaching up to 42 exaFLOPS for sparse FP8 computing, it leverages 10,000 advanced chips. The upcoming models will support eight billion and 70 billion parameters, trained on 15 trillion tokens, and be proficient in over 1,000 languages. In a push for transparency, ETH researchers will release the models, source code, and training data, encouraging innovation across Europe. This initiative aims to foster trust in AI applications, paving the way for responsible, collaborative AI development. The models are expected to be publicly available under an Apache 2.0 license this summer.

Source link