I’ve been exploring OpenAI’s gpt-oss:20b, their first open-source model based on GPT-4, which claims a knowledge cutoff of June 2024. Testing it against a real-world scenario, I used a UK practice test for the 11+ exam, aimed at evaluating selectivity for grammar schools. My RTX 5080, while decent, struggled with the model’s demands, relying heavily on CPU and system RAM.
The first attempt yielded disappointing results—a mere nine correct answers out of 80. The model showcased impressive reasoning but often output irrelevant answers. After increasing the context length to enhance performance, the AI improved slightly, but still produced nonsensical responses.
This experience illuminated the importance of sufficient context for processing complex tasks. While gpt-oss:20b didn’t outperform my 10-year-old, it offered valuable insights into AI capabilities, reinforcing my understanding of model limitations and performance issues. Stay tuned for more updates on AI testing and evaluations.
Source link
