# Google's new TPU systems tackle AI memory and data protection with existing chips

By MarketScale Newsroom · Published 2026-11-14 · Software & Technology on MarketScale
Canonical: https://www.marketscale.com/industries/software-and-technology/googles-new-tpus-tackle-ai-memory-and-data-protection-using-chips-it-already-ships

> Google says TPU 8tx brings confidential computing to TPU 8t silicon, while TPU 8i Max packages two TPU 8i chips together to increase bandwidth.

## Key points

- Neither system uses a new accelerator design: TPU 8tx pairs TPU 8t silicon with fifth-generation AMD EPYC hosts and Google's Boardfly network, while TPU 8i Max packages two existing TPU 8i chips together.
- TPU 8i Max doubles memory bandwidth and on-chip SRAM but raises HBM capacity by 33%, and bandwidth and capacity answer different questions: how fast context data moves versus how much of it the system can hold.
- The 2x latency claim for TPU 8tx is measured against TPU 8t, while the 8i Max gains are measured against TPU 8i. Neither figure says which system finishes a given job more cheaply.

TPU 8i Max packages two existing TPU 8i chips together. A system can change chip packaging, host processors, networking and security without an entirely new accelerator design, according to Zeniteq. TPU 8tx adds confidential computing capabilities built in partnership with AMD, according to Network World.

Earlier this year Google divided its eighth TPU generation into two chips, one built for large-scale training and the other for low-latency inference. The company said that, compared with the previous generation, the chips deliver as much as 2.7 times the training performance and 80% better price performance for inference. Mark Lohmeyer, a vice president and general manager at Google Cloud, told Network World's Taryn Plumb that offering two chips and two systems rather than one was a first for the company, and that the new pair builds on both lines.

Neither system uses a new accelerator design, according to Zeniteq. TPU 8tx pairs the same silicon as TPU 8t with fifth-generation AMD EPYC hosts and Google's low-latency Boardfly network, and TPU 8i Max packages two existing TPU 8i chips together. For the infrastructure lead picking where inference runs, that turns the menu into a question about constraints. One system answers who can see data and model weights while they are being processed. The other answers how fast memory can feed the chip when an agent drags a huge context window behind it.

## Confidential inference on training silicon

According to Network World, TPU 8tx is the first TPU from Google Cloud with confidential computing capabilities, which were developed in partnership with AMD. Lohmeyer said the protection spans the entire path. Data is encrypted on its way from the host to the TPUs, the TPUs stay secured while running inference, and the complete system is attested, so a customer can confirm it is what it claims to be.

Two assets sit inside that boundary at once. One is the enterprise's input data, whether proprietary business records or information it must protect for regulatory reasons. The other is the model's weights, which may be valuable intellectual property belonging to the model owner.

Block Field

In Lohmeyer's view, customers shouldn't be forced to choose between security and performance. Network World reports that Google Cloud carried the low-latency networking design of TPU 8i over to TPU 8tx, which helps cut communication latency between chips. "Because we've reduced the diameter of the network topology, the inference just gets much faster," Lohmeyer said. He added that TPU 8tx gets 2x lower latency than TPU 8t as a result, despite running on the same chip.

The design lines up with where security researchers are pointing. Greg Touhill, director of the CERT Division at Carnegie Mellon University's Software Engineering Institute, told Federal News Network's Terry Gerton that securing AI-based systems and workflows is one of seven areas his group picked for sustained research, alongside secure-by-design engineering across hardware, software and the people who run them. TPU 8tx, which Lohmeyer said adds security directly into the hardware layer, could fit that second category of problem: securing the AI system itself.

## More bandwidth than capacity in 8i Max

Block Field

Read the grid closely and the ratios don't match. Bandwidth and SRAM double, as you'd expect from putting two chips in one package. HBM capacity rises by only a third. Bandwidth measures how much data moves per unit of time, while capacity measures how much data can be held, so doubled bandwidth should not be read as doubled context length.

Lohmeyer framed the design as adding lanes to a highway: individual cars don't go faster, but more traffic gets through at once.

The fit is conditional. For operators running many concurrent agent sessions with very long prompts, doubled bandwidth goes straight at the bottleneck Lohmeyer called the memory wall. For short-prompt chat or classification work that already fits comfortably on a single TPU 8i, the second chip may add cost without adding much.

## The software side of Google's TPU push

On the software side, Google Cloud says llm-d 1.0, an open-source project Google co-founded with partners, is a serving stack built natively for Kubernetes. It links the model servers in a cluster into one coordinated pool, so distributed LLM inference runs as a single system across fleets that mix Google Cloud TPUs and GPUs. Its features include routing that is aware of the cache, offloading of the KV cache across tiers, and flow control based on priority. That suggests Google is attacking the same memory problem from two directions: in software, by spreading KV cache across tiers, and in hardware, through the extra bandwidth of 8i Max.

Porting may also get easier. Google and Meta together open-sourced TorchTPU, a PyTorch stack built natively to bring Google TPUs straight into the PyTorch ecosystem, where they sit next to GPUs and CPUs. Developers can now use frameworks they already know, such as Hugging Face, vLLM and TorchTitan, to train, serve and deploy models on TPUs natively. For teams whose inference code was written for GPUs, that could cut the engineering cost of even trying out a TPU system. For isolation, Google introduced Agent Compute Engine, a new compute service that offers hardware-isolated microVM sandboxes to secure and scale autonomous AI agents.

## Two claims, two different baselines

The strongest objection to reading these systems as a clean either-or is that their headline numbers can't be compared. The 2x latency improvement for TPU 8tx is measured against TPU 8t. The doubled resources in 8i Max are measured against TPU 8i. As Zeniteq points out, neither figure tells a buyer which system will finish a particular workload more cheaply, and both are Google's own claims rather than independent benchmarks.

Block Field

Pricing is the figure a CFO will ask for first, and it will decide whether confidential inference on TPU 8tx becomes the default for regulated workloads or a specialist option reserved for the most sensitive models.

## Sources

- [Google targets AI's next infrastructure bottlenecks with two ...](https://www.networkworld.com/article/4245897/google-targets-ais-next-infrastructure-bottlenecks-with-two-new-tpus.html) (Network World)
- [Google TPU 8tx and 8i Max Target Two AI Bottlenecks](https://www.zeniteq.com/google-tpu-8tx-and-8i-max-target-two-ai-bottlenecks-xltxiz)
- [What's new in AI infrastructure this month](https://cloud.google.com/blog/topics/ai-infrastructure/whats-new-in-ai-infrastructure-this-month)
- [Every cyber threat can't be a top priority](https://federalnewsnetwork.com/cybersecurity/2026/10/every-cyber-threat-cant-be-a-top-priority/) (Federal News Network)
- [Data Center | Network World](https://www.networkworld.com/data-center/)

Tags: Google Cloud, TPU 8tx, TPU 8i Max, AMD EPYC, confidential computing, AI inference, AI infrastructure, data center hardware, cloud procurement, IT operations, AI agents, llm-d

---
Source: MarketScale, https://www.marketscale.com/industries/software-and-technology/googles-new-tpus-tackle-ai-memory-and-data-protection-using-chips-it-already-ships. Published for AI indexing and citation; cite the canonical URL. Site guide for agents: https://www.marketscale.com/llms.txt