Google adds TPU systems for confidential inference and larger working memory Google adds TPU systems for confidential inference and larger working memory

A system redesign around existing silicon

The new names describe system configurations, not a fresh chip generation. TPU 8tx uses the same silicon as TPU 8t, while adopting the Boardfly network associated with TPU 8i. The distinction matters for buyers comparing an accelerator with the system surrounding it. A confidential workload depends on more than the processor design. The host and the connections carrying protected data also belong in that review. In this case, the two additions also split the purchasing question. One asks whether protected information can remain isolated throughout computation. The other asks whether the accelerator can keep enough data moving to sustain a demanding model workload.

The host participates in the security boundary

AMD separately confirmed that its fifth-generation EPYC processors provide the host infrastructure for TPU 8tx. Its account describes the CPU preparing data, coordinating work and communicating with the accelerators. Secure Encrypted Virtualization extends the protection mechanism across that connection. Before an accelerator receives direct access to protected memory, the system verifies its identity and security configuration. AMD says prompts, model weights and intermediate data remain encrypted in CPU memory and during transfers, with isolation during accelerator execution. This makes the host part of the confidential-computing design, rather than merely a feeder for the accelerator. AMD notes that these security features require cloud-provider enablement.

More memory bandwidth serves a different problem

TPU 8i Max puts two TPU 8i chips in one package. Google reports twice the compute, on-chip SRAM and high-bandwidth-memory bandwidth of TPU 8i. The design targets long-context workloads and higher computing density in an existing rack footprint. This is a capacity and data-movement proposition. It does not by itself answer whether an application produces better responses. A coding service, for example, still needs to retrieve the right files and evaluate its output. Faster access to a larger working set changes the infrastructure budget for that service.

The April split still explains the lineup

Google introduced the original TPU 8t and TPU 8i on April 22. The training-oriented 8t emphasized compute throughput and scale-up connections. The inference-oriented 8i emphasized memory bandwidth and communication latency. That earlier design included 288 GB of high-bandwidth memory and 384 MB of on-chip SRAM for TPU 8i. Google also described native support for JAX, PyTorch, SGLang and vLLM across the original platforms. Those choices established a distinction between training a model and serving repeated requests. Today’s configurations add further specialization around that foundation, making the host, memory system and interconnect part of the product decision alongside the accelerator itself.