Google Reconfigures TPU Accelerators for Broader AI Inference Workloads

Google is preparing two modified TPU accelerator variants for deployment in 2027. Rather than introducing entirely new chips, the company is changing how its existing Sunfish TPU 8t and Zebrafish TPU 8i systems are configured to better support different AI workloads.

The original TPU 8t and TPU 8i systems were first revealed in April 2026. Broadcom is managing production of the TPU 8t for Google, and the accelerator is expected to be entering Google’s data centers. MediaTek is managing production of the TPU 8i, which experienced delays earlier in 2026. Its current shipping status is unclear.

The modified Sunfish and Zebrafish systems are expected to begin installation in 2027. Deployment could start in the first quarter, followed by a ramp lasting roughly a year and a half before the machines become available across multiple Google Cloud regions.

TPU availability and deployment

Google has also been expanding access to its previous-generation Ironwood TPU v7p accelerators. These systems were first deployed for Google’s internal use in the fourth quarter of 2025. Google later made them available through Google Cloud, with pricing for the US-Central1 region in Iowa and the Europe-West2 region near London.

It took approximately a year for enough excess Ironwood capacity to become available for cloud customers. The delay illustrates how long it can take for Google to move internally deployed TPU systems into broader commercial availability.

TPU 8t and TPU 8tx

The TPU 8t was designed specifically for AI training. It uses Google’s 3D torus interconnect to connect 9,600 accelerators in a shared coherent memory space. At FP4 precision, a full system delivers 121 exaflops.

Each Sunfish TPU 8t device includes 216 GB of HBM3E memory across six stacks. The scale of this shared-memory configuration is intended to support the training of large frontier models, including Google’s Gemini family.

During a conference call about the new TPU systems, Mark Lohmeyer, Google’s vice president and general manager of AI and computing infrastructure, said that the 9,600-unit pods can be combined into clusters containing more than 1 million TPU 8t devices. According to Lohmeyer, such a cluster was used to train the Gemini 4 Carbon frontier model, which was developed by Google DeepMind and announced on October 28, 2026.

Gemini 4 Carbon followed Gemini 4 Argon by several weeks and performed better across the reported evaluations. Through Google Cloud APIs, both models have introductory pricing of $2 per million input tokens and $10 per million output tokens. Cached input tokens are priced at a 95 percent discount compared with the standard input-token price.

The TPU 8tx is a specialized TPU 8t variant intended primarily for inference, although it can also perform training when otherwise idle. It introduces two major changes.

First, Google replaces the TPU 8t’s 3D torus interconnect with a boardfly interconnect, a variant of the dragonfly topology. Google designed boardfly to provide tighter coupling and lower latency for inference, particularly for complex agentic workloads that run many inference streams simultaneously.

The change reduces pod size from 9,600 accelerators to 1,152. Because the network diameter is 8.3 times smaller, the resulting fabric is flatter. Google says average latency in the smaller system is approximately half that of the 3D torus configuration. The TPU 8tx continues to use Google’s proprietary Inter Chip Interconnect, or ICI.

The second change replaces Google’s Axion Arm server CPU with an unnamed AMD Epyc processor. The AMD processor provides an on-board root of trust and encryption accelerators, allowing workloads on TPU 8tx systems to remain encrypted until processing begins and to be re-encrypted after TPU processing is complete.

Google has indicated that the processor may be a customized Epyc part, although the customization could relate primarily to core count and clock speed. Possible candidates include the current Turin Epyc 9005 or the upcoming Venice Epyc 9006. Google already develops its own security and encryption processors, but the company has not explained why equivalent functionality is not included in the TPU architecture or Axion processors.

TPU 8i Max and larger-context inference

The TPU 8i Max is designed for inference workloads involving larger contexts. Google addresses these workloads by placing two Zebrafish accelerator complexes in each socket. However, total HBM memory capacity per socket increases by only 33 percent compared with the original TPU 8i.

At the chip level, the original Zebrafish TPU 8i used eight HBM3 stacks for a total capacity of 288 GB and aggregate bandwidth of 8.6 TB per second. The TPU 8i Max uses two chips. Each chip provides 192 GB of memory while retaining 8.6 TB per second of bandwidth.

This means each Zebrafish chip uses the same number of HBM3 stacks, but its capacity is reduced by 33 percent. The configuration appears to imply a move from twelve-high HBM3 stacks to eight-high stacks. As a result, the TPU 8i Max has less HBM memory per unit of compute than the TPU 8i, even though its total memory per socket is higher.

Google appears to be distributing the HBM3 allocation across more TPU chips and tightly coupling two Zebrafish chips. This provides a larger shared memory footprint for context data and key-value cache storage in inference models.

The tradeoff could leave some compute capacity unused, but that may be preferable to having insufficient memory per socket for large-context workloads. Google has not explained why it did not reduce compute capacity to balance the memory configuration, or whether MediaTek and Taiwan Semiconductor Manufacturing Co. selected lower-performing Zebrafish chips for this purpose.

The ICI port at the bottom of the TPU 8i Max card also appears different from the corresponding port on other TPU 8 systems. Google has not explained the difference.

Expected pricing

Google has not announced pricing for the Sunfish and Zebrafish variants.

Training-oriented TPUs could command a premium, while inference pricing remains important as AI providers seek to reduce the cost of processing tokens. The TPU 8t could cost more per unit of compute and HBM capacity than the TPU 8i. The TPU 8tx could carry a modest hourly premium over the TPU 8t and TPU 8i, while the TPU 8i Max could cost somewhat less than twice the hourly price of the TPU 8i.

These remain expectations until Google publishes official pricing.

TPU Sync and memory utilization

Google has also developed TPU Sync, a hierarchical storage management software layer intended to improve HBM utilization during sporadic and bursty agentic AI workloads.

Lohmeyer said that TPUs are often idle for a significant portion of their operating time. Keeping those resources reserved while an agent is paused can reduce system efficiency, particularly as agentic workloads grow.

TPU Sync moves an agent’s working memory off the TPU when the agent pauses and restores it when processing resumes. This can release accelerator resources for other tasks, increasing overall utilization and potentially reducing the cost per transaction or agentic activity.

Because TPU Sync is software-based, Google can support earlier TPU generations. The company has extended it to TPU 5, TPU 6, TPU 7, and TPU 8 systems.

The mechanics of coordinating memory movement across clusters containing tens of thousands or millions of TPUs remain unclear. The data may need to move among HBM, host DRAM, host flash, and external flash or disk storage. Google has not disclosed how much TPU Sync improves inference throughput, or how the system coordinates these transfers at very large cluster sizes.