A recent technical paper, “System-performance and cost modeling of Large Language Model training and inference,” authored by imec researchers, explores the scalability challenges posed by large language models (LLMs) using transformer architectures. While LLMs have transformed various AI fields, issues such as increased model size and complexity often outstrip computing power and memory capabilities. To tackle these challenges, the paper presents a novel performance-cost modeling methodology that employs advanced compute techniques, memory optimizations, and innovative communication strategies. Key innovations like flash attention and mixture of experts models are integrated to alleviate performance bottlenecks. The framework examines the effects of diverse network topologies and communication algorithms, incorporating a chiplet cost model for in-depth analysis. This methodology aims to enhance future compute system designs and foster effective hardware-software co-development by assessing performance-cost trade-offs across different architectural configurations. For detailed insights, access the technical paper here.
Share
Read more