Barclays projects that consumer AI inference chip-related capital expenditure will reach nearly $120 billion by 2026, soaring past $1.1 trillion by 2028. Leading AI firms, including OpenAI, are pivoting towards custom chips, particularly Application-Specific Integrated Circuits (ASICs), to enhance profitability by lowering inference costs. Google TPUs (Tensor Processing Units) are highlighted for their cost-effectiveness in this landscape, as they significantly reduce OpenAI’s compute budget—more than 50% is spent on inference alone. Despite their older models lacking peak performance compared to Nvidia GPUs, TPUs offer lower cost-per-inference due to their efficient architecture, minimizing energy waste. Omdia’s Alexander Harrowell supports this view, noting that many AI professionals achieve a superior ratio of floating-point operations per second (FLOPS) from TPUs compared to other options. As the AI industry evolves, the shift toward dedicated chips like TPUs signifies a crucial strategy to maintain profitability.
Source link
