The author reflects on their experience with NetHack as a complex benchmark for reinforcement learning (RL) and agentic AI. They initially explored the game personally, accumulating insights through extensive play, which culminated in their character achieving a high-level status. Key challenges identified include:
- Credit Assignment: Agents must accurately track rare yet crucial events impacting gameplay, like acquiring resistances.
- In-Context Exploration: Identifying unknown magical objects requires careful observation and piecing together clues amid randomized attributes.
- Leveraging Offline Knowledge: Players use resources like the NetHack Wiki, highlighting the importance of integrating cultural knowledge into AI training.
- Combinatorial Complexity: Success relies on generalizing strategies across various game scenarios presenting unique challenges.
- Hierarchy in Decision-Making: The game supports multi-level hierarchies, suggesting the need for hierarchical RL approaches.
Overall, NetHack serves as a rigorous testing ground for AI capabilities while being an engaging experience itself.
