最新 TPU 8tx 与 8i Max:为智能体时代打造

高级副总裁兼首席技术专家,AI 与基础设施
我们正在扩展第八代 TPU 产品组合,推出全新的 TPU 8tx,用于安全、保密的推理,以及 TPU 8i Max,用于超大规模上下文窗口。这些系统通过为安全性、密度和推理提供专用硬件,帮助你管理复杂的智能体式工作负载。你可以观看我们的直播,或访问 Google Cloud 网站了解我们完整的基础设施产品线,从而进一步了解这些新增产品以及我们最新的软件更新。
摘要由 Google AI 生成。生成式 AI 仍处于实验阶段。
我们正在扩展第八代 TPU 产品组合,新增 TPU 8tx 和 TPU 8i Max 系统。
TPU 8tx 为敏感数据提供硬件级安全保护,而 TPU 8i Max 则负责处理超大规模上下文窗口。
这些系统与 TPU 8t 和 TPU 8i 一同,为 AI 生命周期的每个阶段提供专门的性能支持。
我们通过支持 PyTorch、开源软件以及面向开发者的自动化工具,让 TPU 更易于使用。
摘要由 Google AI 生成。生成式 AI 仍处于实验阶段。
我们最初设计 TPU 是在十多年前,动因来自 Google 的内部需求,包括语音识别和语言翻译。历经八代 TPU 之后,应用场景已呈爆发式增长,涵盖:i)训练并服务我们自己的 Gemini 模型;ii)智能体式工作负载;iii)在我们的云基础设施上向 Google Cloud 客户交付服务;以及最近的 iv)允许客户将 TPU 直接部署到自己的数据中心。如今 TPU 是一项快速增长的业务,拥有近 100 家客户——包括前沿实验室、交易公司、零售商、大学等。
在 4 月的 Google Cloud Next 上,我们在同一代产品中推出了两款新的专用 TPU——用于大规模训练的 8t 和用于实时推理的 8i。与上一代 TPU 相比,TPU 8t 和 8i 分别带来最高 2.7 倍的训练性能提升和 80% 的推理性价比提升。
今天,我们更进一步,以两款新系统扩展第八代 TPU 产品组合:面向超大上下文窗口和最高单机架算力密度的 TPU 8i Max,以及面向安全机密推理的 TPU 8tx。这是我们首次在一年内推出四款 TPU 系统。
我们扩展后的第八代 TPU 产品组合:TPU 8t、TPU 8i、TPU 8tx 和 TPU 8i Max。
虽然我们一直与 Google DeepMind 合作共同设计 TPU,但如今我们越来越多地从外部机构获取关于其具体需求的反馈。随着计算从人类规模迈向智能体规模,企业需要能够在 AI 模型庞大的上下文窗口中进行推理的加速器。此外,一些受高度监管的客户正为特定工作负载寻求专门的加密能力。
我们今天发布的新 TPU 系统正是为应对这一新现实而生。结合我们近期在 PyTorch 方面的发布,我们的目标是让每一位开发者和云客户都能更便捷地使用 TPU。通过将模块化的定制芯片与开放、易用的软件栈相结合,我们能够以 AI 行业的速度交付新的硬件和系统配置。
围绕芯片构建计算机
今天,我们发布两款全新的 TPU 系统——而新芯片与新系统之间存在着重要区别。
设计一款新芯片,意味着要在硅晶圆上排布数十亿个微观晶体管,这一过程耗时数年,既受物理定律约束,也受制于复杂而漫长的制造生命周期。而系统则关乎我们如何把芯片周边的计算、网络和存储整合在一起。这其中可以包括它们与芯片的封装方式,以便快速满足新兴的客户需求和研究用例。
十多年前我们打造第一颗 TPU 时就知道,AI 软件的演进速度会远超芯片制造工艺所能达到的速度。因此我们在系统均衡性和模块化上做出了根本性的押注,将 TPU 设计为 AI Hypercomputer 的组成部分、作为可拼装的积木,让工程师能够围绕它们快速调整封装、高速内存、光网络和安全配置。这种方式让我们能在几个月而非几年的时间里交付专用超级计算机。
面向安全、高上下文 AI 的专门化设计
我们第八代 TPU 产品组合中的每一款系统都针对 AI 生命周期中一个不同的环节。TPU 8t 和 TPU 8i 奠定了这一代芯片的基线:
TPU 8t 被设计为面向大规模基础模型的训练利器,追求最大化的“goodput”——即每瓦能耗所产出的有效智能。
TPU 8i 则是推理引擎,旨在突破内存墙,将芯片间通信延迟最多缩短一半。
如今,随着企业将自主智能体投入生产环境,我们为这些基础系统打造了两种全新配置,以解决两个迫在眉睫的新挑战:硬件级数据安全与百万级 Token 上下文规模。
TPU 8tx 是我们的第三款第八代系统,针对安全保密推理进行了优化,同时也很适合小规模训练。它面向的是那些在自有最敏感数据旁运行前沿模型或专有模型以及自主智能体的受监管企业和政府机构。
通常情况下,数据在存储时或通过网络传输时会被加密,但在计算机处理数据时必须在内存中解密。TPU 8tx 与 TPU 8t 基于同一芯片打造,并搭配第五代 AMD EPYC CPU 主机,将安全性直接构建在硬件层,即使在数据处于活跃使用状态时也能保护并隔离敏感数据和模型权重。同时,通过采用我们为实时推理打造的同一套低延迟 Boardfly 网络,TPU 8tx 能让多步智能体持续运转,相比 TPU 8t 延迟最多降低 2 倍,而性能损失几乎为零。
将训练级算力与高速推理网络相搭配,还意味着企业可以在单一系统上同时运行并定制自己的模型。TPU 8tx 让团队能够利用私有数据微调模型、将大模型蒸馏为小模型,并通过强化学习训练自己的智能体,而无需在彼此独立的集群之间迁移敏感数据。
TPU 8tx 专为敏感的企业工作负载设计,为私有微调与安全的智能体执行提供硬件级机密计算。
我们的第四款系统 TPU 8i Max,则是为另一个不同的问题而打造:内存墙。
当数十个 AI 智能体同时在整个软件代码库或数小时的视频上协同工作时,内存带宽很快就会成为瓶颈。因此,我们把两颗低延迟的 TPU 8i 芯片组合到同一个封装中。
TPU 8i Max 让算力与内存同步扩展,提供两倍于 TPU 8i 的算力、两倍的 SRAM 和两倍的高带宽内存(HBM)带宽,使处理器始终配备关键数据,大型工作负载不会停滞。此外,由于 TPU 8i Max 在现有 机架 空间内放入了两倍的硅面积,数据中心每平方英尺、每瓦可部署的算力大幅提升。
双芯片的 TPU 8i Max 系统,专为超大规模上下文窗口和最大算力密度而设计。
我们的第八代 TPU 中的这四款系统虽然在不同工作负载上都具备灵活性,但它们共同为客户提供了针对智能体生命周期每个阶段专门打造的加速器:TPU 8t 用于纵向扩展训练,TPU 8i 用于低延迟推理,TPU 8tx 用于机密计算,TPU 8i Max 用于超大规模上下文窗口和每机架最高算力密度。
向每一位开发者开放 TPU
十多年前我们推出第一代 TPU 时,只有少数几家公司运行着需要 TPU 处理能力的工作负载。当时的用例都很专门化,因此工程师会使用经过优化的框架和手工调优的代码。
多年来,随着 AI 的进步,越来越多的公司开始使用加速器。如今,AI 的应用已经发展到这样一个阶段:TPU 对众多公司和用例而言都表现出色,因此,我们一直在系统性地消除应用障碍,让 TPU 像通用硬件一样易于使用。
这项投入如今在这些关键领域汇聚成形:
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Our newest TPUs: 8tx and 8i Max, built for the agentic era)
SVP and Chief Technologist, AI and Infrastructure

We’re expanding our eighth-generation TPU portfolio with the new TPU 8tx for secure, confidential inference and TPU 8i Max for massive context windows. These systems help you manage complex agentic workloads by providing specialized hardware for security, density, and reasoning. You can learn more about these additions and our latest software updates by watching our broadcast or visiting the Google Cloud website to explore our full infrastructure lineup.
Summaries were generated by Google AI. Generative AI is experimental.
We are expanding our eighth-generation TPU portfolio with the new TPU 8tx and TPU 8i Max systems.
TPU 8tx provides hardware-level security for sensitive data, while TPU 8i Max handles massive context windows.
These systems join TPU 8t and TPU 8i to offer specialized performance for every AI lifecycle stage.
We are making TPUs more accessible by supporting PyTorch, open-source software, and automated tools for developers.
Summaries were generated by Google AI. Generative AI is experimental.
We first designed TPUs more than a decade ago motivated by Google’s internal demands, including voice recognition and language translation. Eight generations of TPUs later, the use cases have exploded, covering: i) training and serving our own Gemini models, ii) agentic workloads, iii) delivering services to Google Cloud customers on our cloud infrastructure, and most recently, iv) allowing customers to deploy TPUs directly into their own data centers. TPUs are now a rapidly growing business with nearly 100 customers — including frontier labs, trading firms, retailers, universities, and more.
At Google Cloud Next in April, we introduced two new specialized TPUs in the same generation — 8t for large-scale training and 8i for real-time reasoning. TPU 8t and 8i offer up to 2.7 times better training performance and 80% better inference price performance, respectively, versus our previous generation of TPUs.
Today, we’re taking that a step further, expanding our eighth-generation TPU portfolio with two additional systems: TPU 8i Max for massive context windows and the highest compute density per rack, and TPU 8tx for secure and confidential inference. This is the first time we have introduced four TPU systems in a single year.
Our expanded eighth-generation TPU portfolio: TPU 8t, TPU 8i, TPU 8tx, and TPU 8i Max.
While we’ve always co-designed our TPUs in partnership with Google DeepMind, we’re now increasingly getting feedback from external organizations on their specific needs. With computing moving from human to agent scale, businesses are asking for accelerators that can reason across AI models’ massive context windows. In addition, some highly regulated customers are asking for specialized encryption capabilities for unique workloads.
The new TPU systems we are announcing today address this new reality. Coupled with our recent PyTorch announcements, our aim is to make TPUs more easily accessible to every developer and Cloud customer. By combining modular custom silicon with an open and easy-to-use software stack, we can deliver new hardware and system configurations at the pace of the AI industry.
Building the computer around the chip
Today, we’re announcing two new TPU systems — and there’s an important difference between a new chip and a new system.
Designing a new chip means laying out billions of microscopic transistors on a silicon wafer in a multi-year process bound by the laws of physics and a complex, long-duration manufacturing lifecycle. A system, on the other hand, is about how we bring together the compute, networking, and storage around the chip. This can include how they are packaged with the chips, to quickly meet emerging customer needs and research use cases.
When we built our very first TPU more than a decade ago, we knew AI software would evolve much faster than silicon manufacturing ever could. So we made a foundational bet on system balance and modularity, architecting our TPUs as building blocks as part of our AI Hypercomputer, so engineers could rapidly adjust packaging, high-speed memory, optical networking, and security around them. That approach allows us to deliver specialized supercomputers in a matter of months rather than years.
Specializing for secure, high-context AI
Each system in our eighth-generation TPU portfolio targets a distinct part of the AI lifecycle.
TPU 8t and TPU 8i established the baseline for this generation of chips:
TPU 8t was designed to be a training powerhouse for massive foundation models with maximum “goodput” — or the useful intelligence delivered per watt of energy used.
TPU 8i is the reasoning engine, designed to break the memory wall, cutting inter-chip communication delays by up to half.
Now, as enterprises bring autonomous agents into production, we have created two new configurations of these foundational systems to solve two urgent new challenges: hardware-level data security and million-token context scale.
TPU 8tx, our third eighth-generation system, is optimized for secure and confidential inference and is also great for small-scale training. It’s built for regulated enterprises and governments running frontier or proprietary models and autonomous agents alongside their most sensitive data.
Normally, data is encrypted when stored or sent over a network, but has to be unlocked in memory while a computer processes it. Built on the same silicon as TPU 8t and paired with 5th Generation AMD EPYC CPU hosts, TPU 8tx builds security directly into the hardware layer, keeping sensitive data and model weights protected and isolated even while in active use. And by adopting the same low-latency Boardfly network we created for real-time reasoning, TPU 8tx keeps multi-step agents moving, delivering up to 2x lower latency compared to TPU 8t with a near-zero performance penalty.
Pairing training-grade compute with a fast inference network also means organizations can simultaneously run and customize their models on a single system. TPU 8tx enables teams to fine-tune models using their private data, distill large models into smaller ones, and train their agents through reinforcement learning, all without needing to migrate sensitive data between separate clusters.
Designed for sensitive enterprise workloads, TPU 8tx provides hardware-level confidential compute for private fine-tuning and secure agent execution.
Our fourth system, TPU 8i Max, is built for a different problem: the memory wall.
When dozens of AI agents work together across an entire software codebase or hours of video at once, memory bandwidth quickly becomes the bottleneck. So we took two low-latency TPU 8i chips and combined them onto a single package.
TPU 8i Max scales compute and memory in lockstep, delivering double the compute, double the SRAM and double the HBM bandwidth of TPU 8i, so processors remain equipped with essential data and so that large workloads do not stall. Further, because TPU 8i Max fits twice the silicon within the existing rack footprint, data centers can deploy significantly more computing power per square foot and per watt.
The dual-chip TPU 8i Max system, engineered for massive context windows and maximum compute density.
While flexible across workloads, together the four systems in our eighth-generation TPUs give customers a purpose-built accelerator for every stage of the agentic lifecycle. TPU 8t for scale-up training, TPU 8i for low-latency reasoning, TPU 8tx for confidential computing, and TPU 8i Max for massive context windows and the highest compute density per rack.
Opening up TPUs to every developer
When we launched the first TPU more than a decade ago, there were only a handful of companies running workloads that needed the processing power of a TPU. The use cases were specialized, so engineers would use optimized frameworks and hand-tuned code.
Over the years, as AI has advanced, more and more companies began using accelerators. Now, AI adoption is at a point where TPUs are excellent for a wide range of companies and use cases, and as a result, we’ve been working systematically to remove barriers to adoption so that TPUs are as easy-to-use as general-purpose hardware.
This investment comes together today across these key areas:
Opening up TPU development to every PyTorch developer; no code rewrites needed: Last month, we open-sourced TorchTPU, so PyTorch code and Hugging Face models now run natively on TPUs, alongside out-of-the-box TPU backends for leading open-source inference engines like vLLM and SGLang. And to make hardware optimization effortless, our new open-source TPU agent suite (now in preview) plugs directly into coding tools like Claude Code, Cursor, and Google Antigravity to automate TPU onboarding in minutes, diagnose bottlenecks, and autotune custom kernels using plain language. For production deployments, our new open-source XLA Serving Engine lets you train models in Python and serve them directly in a low-latency C++ runtime across TPUs, GPUs, and CPUs.
Industry-leading production performance on InferenceX and AgentX benchmarks:
In production, what really matters is outcomes on real AI workloads rather than theoretical peak chip speeds. On industry-standard InferenceX and AgentX benchmarks, which test realistic agentic coding and long-context work, our native vLLM-TorchTPU software stack on TPUs achieved state-of-the-art price-performance marks. New evaluations demonstrate these advantages extend across a wide range of frontier open models and open-source serving stacks while maintaining high model accuracy. Upcoming software optimizations on this same hardware will drive those results even further.
Eliminating idle compute waste for autonomous agents: Because AI agents are stateful, they frequently pause for seconds or hours while waiting on a database lookup, an external software tool, or human feedback. To make our accelerators more power-efficient during those pauses, our new Agent Compute Engine software automatically suspends idle agent sessions and shifts working memory off the chip in milliseconds, cutting operational infrastructure costs.
Breaking through the memory wall with the model-state layer (TPU Sync): Building the computer around the chip also means unifying memory, storage, and networking in software. As agents reason across million-token contexts and continuous reinforcement learning (RL) loops, memory quickly becomes the bottleneck. We’re open-sourcing TPU Sync, the model-state layer for TPU clusters. TPU Sync turns chip HBM, host DRAM, shared storage, and the data center network into a single, co-designed memory hierarchy — automatically tiering massive KV caches off the chip in milliseconds, streaming context across disaggregated prefill and decode nodes, and synchronizing RL weights to cut KV transfer latency and Time to First Token (TTFT) by more than 90%.
Scaling distributed workloads across TPUs from a single controller: Now generally available on Google Kubernetes Engine (GKE), Cloud Pathways brings the Google DeepMind runtime behind Gemini to every Google Cloud customer. Using a single-controller architecture and pathwaysutils, you can orchestrate large-scale TPU systems across multiple slices as if they were one local machine for pre-training, reinforcement learning, and multihost inference. Cloud Pathways keeps long-running jobs moving through hardware disruptions with colocated Python sidecars, sub-minute slice hot-swapping, and elastic resharding.
Stitching custom TPU topologies on-demand with GKE dynamic slicing: Now generally available, GKE dynamic slicing decouples physical hardware provisioning from logical slice formation using optical circuit switching (OCS) technology. By dynamically assembling TPU cubes into larger topologies (super-slicing) or partitioning them into smaller, isolated workloads (sub-slicing) on-the-fly, this update completely eliminates rigid, manual nodepool recreation cycles. This topology agility slashes job startup times by up to 5x and accelerates fault recovery by up to 4.5x, maximizing overall fleet utilization and workload goodput across all TPUs.
Co-designing with a decade of experience
Shipping four TPU systems and a unified, open software stack in a single year only happens when silicon architects, networking engineers, compiler teams, and Google DeepMind researchers build side-by-side.
Increasingly, we’re even tapping into AI to ideate and build the next generation of AI infrastructure. From using Google DeepMind’s AlphaChip reinforcement learning system to using Gemini to deliver chip layouts in hours rather than weeks, we’re using TPU-powered AI services to rapidly accelerate next-generation TPU hardware development. This is true co-design and a unique Google differentiator.
Learn more about our new TPU systems by tuning into today’s broadcast at 9 a.m. PT.
原文为英文,正文由自动抓取并保留原文。
原文发布于 Google Blog(Our newest TPUs: 8tx and 8i Max, built for the agentic era),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。