OpenAI has unveiled its first open-weight language models since GPT-2, named GPT-OSS, featuring two sizes: a 117 billion parameter model and a 21 billion parameter version. Both models are available under a permissive Apache 2.0 license, allowing extensive usage without restrictions. The training focused on English text, emphasizing STEM, coding, and general knowledge, though they lack the vision capabilities of larger models like GPT-4.
Notably, the models utilize a mixture of expert (MoE) architecture, enhancing token generation speed. The 120B model can run on a single 80GB H100 GPU, while the 20B variant fits in just 16GB of VRAM. Although they support a context window of 128K tokens, competition exists with models offering larger contexts.
OpenAI has prioritized safety, implementing measures to filter harmful data and inviting developers to identify potential vulnerabilities. GPT-OSS is now accessible on various repositories, including Hugging Face.
Source link
