Key Highlights from Hot Chips 2025: Noam Shazeer’s Keynote on AI’s Future
At Hot Chips 2025, Noam Shazeer, co-lead at Google Gemini and co-author of the influential transformer paper “Attention Is All You Need,” delivered a compelling keynote titled “Predictions for the Next Phase of AI.” He emphasized that language modeling represents a critical challenge, suggesting that as AI models evolve, an increasing number of FLOPS (floating-point operations per second) are essential for enhanced performance. Shazeer noted the transformation in AI training, revealing that while 32 GPUs were pioneering a decade ago, today’s models utilize hundreds of thousands.
He detailed the hardware requirements for advanced AI, highlighting the need for greater compute, memory capacity, and bandwidth at all levels, including HBM and on-chip SRAM. Ultimately, Shazeer asserted that scaling computational resources is vital for developing superior large language models (LLMs), making a strong case for continuous advancements in AI technology.
