Key Highlights
- Google revealed its eighth-generation tensor processing units: TPU 8t for model training and TPU 8i for inference operations
- The inference-focused TPU 8i achieves 80% improved performance-per-dollar compared to the Ironwood predecessor
- Broadcom partnered with Google on chip development, with design input from Google DeepMind
- The TPU 8t training processor supports configurations up to 9,600 chips and delivers double the interchip bandwidth of Ironwood
- Google Cloud customers will gain access to both processors before the end of this year
Google introduced two distinct custom AI processors on Wednesday, representing the first time the company has separated its tensor processing unit architecture into dedicated training and inference hardware.
The eighth-generation lineup consists of the TPU 8t for AI model training workloads and the TPU 8i engineered specifically for inference tasks — deploying trained models in live production environments. Broadcom collaborated on the development of both processors, extending a technology partnership that spans more than ten years.
This release represents a strategic evolution in chip architecture. Earlier TPU generations combined training and inference capabilities within a single processor design. Google attributes this architectural split to the growing requirements of agentic AI systems, where models execute in persistent loops with minimal human oversight.
“With the rise of AI agents, we determined the community would benefit from chips individually specialized to the needs of training and serving,” said Amin Vahdat, Google’s SVP and chief technologist for AI and infrastructure.
The TPU 8i inference processor incorporates 384 megabytes of SRAM per chip — three times the memory capacity available in Ironwood. According to Google, this expanded memory capacity resolves what the company describes as the “waiting room” effect, where latency accumulates when numerous users simultaneously query a model.
Inference Capabilities See Substantial Performance Boost
The TPU 8i provides 80% superior performance-per-dollar relative to Ironwood. This translates to managing approximately double the workload volume while maintaining identical operational costs.
The chip also achieves up to twice the performance-per-watt efficiency, enabled by integrated power management systems that dynamically adjust power consumption based on real-time demand.
Both processors now operate on Google’s Axion CPU host platform for the first time, enabling system-wide optimization beyond individual chip performance enhancements.
Regarding training capabilities, the TPU 8t superpod architecture supports configurations reaching 9,600 chips and 2 petabytes of high-bandwidth memory. The interchip bandwidth doubles that of Ironwood, and Google indicates the system can compress frontier model development timelines from months down to weeks.
The training processor also provides 2.8 times the computational performance of the seventh-generation Ironwood at an equivalent price point.
Early Adopters and Implementation Partners
Adoption momentum continues building. Citadel Securities developed quantitative research platforms using Google’s TPU infrastructure. The complete network of 17 U.S. Department of Energy national laboratories operates AI co-scientist applications on the processors. Anthropic has pledged to utilize multiple gigawatts of Google TPU computing capacity.
DA Davidson analysts projected in September that the combined valuation of the TPU business and Google DeepMind could reach approximately $900 billion.
Google maintains an exclusive distribution model for TPUs — the processors are accessible solely through Google Cloud services rather than external sales. Nvidia continues serving as Google’s GPU chip supplier, and Google confirmed its position among the initial cloud providers delivering Nvidia’s forthcoming Vera Rubin platform later this year.
Google DeepMind also participated in the chip design process, having utilized the processors for training Gemini models and powering algorithms that drive Search and YouTube services.
Google announced that both the TPU 8t and TPU 8i will reach general availability for cloud customers during the latter part of this year.
