Together 存储加速器:GPU 集群快速免费数据缓存

推出 Polymorphic Inference,首先带来 Together Compiler →
推出 Together Post-Training:更好的模型、更快的运行、更低的成本 →
推出 Together Link:在你已在使用的 harness 中运行开放模型 →
开源 AI 技术栈:模型、推理、路由器、harness 与工具 →
GLM-5.3 与 GLM 5.3 Flash 在 DeepSWE 上的对比 →
⚡ 按需 B200 现已在 Together GPU 集群上线 →
以 API 形式提供的高性能推理
面向批处理工作负载的推理
带 SLA 的基于 Token 的容量
专属模型推理
基于定制硬件的推理
专属容器推理
面向定制模型的推理
在您的测试框架中运行开放模型
随时间不断优化的推理
探索顶级开源模型
可靠的大规模 GPU 集群
前沿规模的自定义基础设施
为 AI 构建开发环境
安全存储模型权重与数据
定义你自己的 RL 和 SFT 循环
启动托管式 SFT 和 DPO 作业
微调顶级开源模型
面向生产级 AI 的系统研究
我们所有的研究出版物
Together AI 技术文档
我们的最新新闻与博客文章
我们的开源演示应用
实用实施指南
用开放模型打造更好的产品
在你的评测框架中运行开放模型
浏览我们的活动日历
开源 ROI 计算器
以 API 形式提供的高性能推理
面向批处理工作负载的推理
带 SLA 的基于 Token 的容量
专用模型推理
在定制硬件上进行推理
专用容器推理
面向自定义模型的推理
在你的运行框架中运行开放模型
随时间不断优化的推理
探索顶级开源模型
大规模且可靠的 GPU 集群
前沿规模的自定义基础设施
为 AI 构建开发环境
安全存储模型权重与数据
定义你自己的 RL 与 SFT 循环
启动托管式 SFT 和 DPO 任务
微调顶尖开源模型
面向生产级 AI 的系统研究
我们的全部研究论文
Together AI 技术文档
我们的最新新闻与博客文章
我们的开源演示应用
实用实施指南
用开放模型构建更好的应用
在你的运行框架中运行开放模型
浏览我们的活动日历
开源 ROI 计算器
推出 Together Storage Accelerator:为你的 GPU 集群提供快速、免费的数据缓存
每个 GPU 节点上都部署分布式缓存:热读取速度提升约 20 倍,每个集群只需从源站拉取一次数据,且无需改动应用。随 Together GPU Clusters 免费提供。
你的 GPU 有多快,取决于你的数据管道有多快。每一次训练运行都始于 30 分钟的模型下载,每个节点都要从全国各地重新拉取同一份数据集,每一次 Hugging Face 的速率限制都会让任务停滞——这一切都让算力资源闲置着等待数据,而你却在为它们付费。
今天我们推出 Together Storage Accelerator(TSA),这是面向 Together GPU Clusters 的分布式缓存层,可免去从对象存储和 Hugging Face Hub 重复读取数据的需求,在显著加速 AI 数据管道的同时降低数据出网费用。现在,每个 Together GPU Cluster 都免费包含 TSA。
问题所在:你的数据离你的 GPU 很远
大多数 AI 团队把数据的唯一真实来源放在对象存储(Amazon S3、Cloudflare R2、Together AI 的 T4*)中,或直接从 Hugging Face Hub 拉取模型。这意味着每次冷启动都要交一笔 WAN 税:跨云或跨区域读取速度慢、成本高(从 AWS 出站的流量费用最高可达 0.09 美元/GB),而且会受到限流。当数百个节点同时拉取同一份 checkpoint 时,这种浪费会成倍放大,并在源站对你限流时让你付出时间、金钱和任务失败的代价。
*T4 是 Together AI 面向数据中心的 S3 兼容对象存储解决方案,目前处于有限预览(Limited Preview)阶段。如果你有兴趣试用,请联系销售或你的客户团队。
TSA 缓存数据,提升性能、可靠性与成本表现
TSA 为 GPU 集群提供了一层分布式缓存。它运行在你集群中的每一个 GPU 节点上,并为你已经在使用的存储提供可直接替换的接入端点。任何节点首次读取某个对象时,TSA 会将其缓存到集群的 NVMe 上。因此,之后来自任意节点的每一次读取,都由集群内部提供服务,本地读取速度大幅提升。
以下是基于 Together GPU 集群的 H100 节点测得的結果:
热读取速度约为从跨云源站拉取的 20 倍,每节点最高可达 3.4 GB/s。一个 2 GiB 的对象从 Cloudflare R2 下载需要 13-15 秒,而从缓存读取只需 0.6 - 0.7 秒
冷读取不产生额外成本:通过 TSA 进行的首次读取,其表现与直接下载相差在约 5% 以内
每个集群只需回源一次。200 个节点拉取一个 70 GB 的 checkpoint 时,只从源站下载一次,而不是 200 次
对故障和速率限制的韧性。即使上游无法提供服务,缓存数据仍能持续提供响应。
你无需对应用做任何改动即可开始使用 TSA。不需要新的 SDK,不需要重写数据加载器,一条命令就能开始使用。
# 兼容 S3 的存储(S3、R2、T4 等)export AWS_ENDPOINT_URL=http://tsa.tsa.svc.cluster.local:18080 # Hugging Face Hubexport HF_ENDPOINT=http://tsa.tsa.svc.cluster.local:18080
你现有的 boto3 代码、transformers 调用和 CLI 工作流都能原样运行。TSA 使用你配置的凭证完成身份认证。凭证会被加密并安全存储在密钥保险库中。
测量于 2026-10-28,测试点为 Together GPU Cluster(us-central)中一个 H100 节点上的 pod。除特别说明外均为单流读取;客户端写入 /dev/null 或节点 NVMe。
从跨云存储桶(Cloudflare R2)读取一个 2 GiB 对象:不使用 TSA 时为 143–169 MB/s(12.7–15.0 秒),TSA 预热后为 3.0–3.4 GB/s(0.64–0.72 秒)。约 20 倍
从同一存储桶读取小文件(5 KB、2.4 MB):不使用 TSA 时为 320 ms 和 680 ms,TSA 预热后为 5 ms 和 8 ms。延迟降低约 60–80 倍
200 节点集群拉取同一个检查点:未使用 TSA 时回源拉取 200 次,使用 TSA 时拉取 1 次。出口流量减少 200 倍
在 Together GPU Clusters 预览部署(TSA chart 0.6.0)上测得;你的结果取决于源站、对象大小和网络路径。源站速度因区域而异:同一个 Hugging Face 下载在 us-central 比在 ca-west 快 5 倍,因此源站越慢,TSA 带来的收益越大。方法:从 GPU 节点上的 pod 中使用单流 curl 和 aws s3 cp,重复读取同一对象,并通过 MD5 与直接下载进行校验。
TSA 利用 ASTRA(Accelerated Storage Transport & Retrieval Architecture),这是我们为支撑全球一些最大规模 GPU 集群(包括我们自己的推理集群)而构建的分布式缓存技术。我们将在一篇即将发布的工程文章中更深入地介绍 ASTRA。
TSA 可与任何兼容 S3 的存储配合使用,并与 T4(Together AI 的托管对象存储解决方案)天然搭配。今天即可将 TSA 指向你现有的 S3 或 R2 存储桶,无需迁移数据。当你希望温数据永久存放在 GPU 附近时,只需更改一项配置即可使用 T4。T4 目前处于限量预览阶段:请联系销售或你的客户团队获取访问权限。
Together Storage Accelerator 现已在 Together GPU Clusters 上可用,且无需额外费用。你可以在集群的设置页面启用,或查阅文档,在十分钟内从零完成加速配置。
推出 Together Storage Accelerator:为你的 GPU 集群提供快速、免费的数据缓存
Together GPU Clusters 自动修复:从 GPU 故障到健康节点
推出可抢占计算:同样的算力,一半的价格
Together AI 与 Y Combinator 达成合作,为 YC 社区推出首个专属 GPU 集群
Together GPU Clusters 新功能:为生产级 GPU 集群带来可靠性与控制力
无冲突的容量:面向 AI 原生团队的多租户 GPU 集群设计指南
Together GPU Clusters 新功能:自动扩缩容、可观测性与自愈能力
深入多节点训练:如何跨 GPU 集群扩展模型训练
Together Instant Clusters 正式全面可用,提供开箱即用、自助式的 NVIDIA GPU
将 100,000 块 GPU 带到欧洲
Together AI 在 GTC 为先锋者提供动力:NVIDIA Blackwell GPU、Instant GPU 集群,以及面向 AI 创新的全栈方案
推出由 NVIDIA GPU 加速的 Together Instant GPU 集群,几分钟内即可自助开通
借助 Together Kernel Collection 大幅提升 NVIDIA H200 和 H100 GPU 集群性能
用于测试和运行大型 GPU 集群以训练生成式 AI 模型的实践者指南
算力增长至 20 exaflops,Together GPU 集群助力初创企业和大型企业加速生成式 AI 开发
为 GPU 集群优化训练工作负载
推出 Together Storage Accelerator:为你的 GPU 集群提供快速、免费的数据缓存
去年我们构建了巨型内核,今年我们把它们删掉了:通往 Together Compiler 之路
让开放模型胜任长时程智能体式工作负载
我们在 Together AI 如何做智能体评估
Together Compiler:面向 AI 推理的持续编译
从静态到多态推理:生产级开源推理系统的新时代
Together GPU 集群自动修复:从 GPU 故障到健康节点
Together Post-Training:前沿级模型塑造,以适合你的方式
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Introducing Together Storage Accelerator: fast, free data caching for your GPU cluster)
Introducing Polymorphic Inference, starting with Together Compiler →
Introducing Together Post-Training: Better models, faster runs, lower costs →
Introducing Together Link: Run open models inside the harness you already use →
The Open Source AI Stack: models, inference, routers, harnesses & tools →
GLM-5.3 vs. GLM 5.3 Flash on DeepSWE →
⚡ On-demand B200s now available on Together GPU Clusters →
High-performance inference as APIs
Inference for batch workloads
Token-based capacity with SLAs
Dedicated Model Inference
Inference on custom hardware
Dedicated Container Inference
Inference for custom models
Run open models in your harness
Inference that gets better over time
Explore Top Open-Source Models
Reliable GPU clusters at scale
Custom infrastructure at frontier scale
Build development environments for AI
Store model weights & data securely
Define your own RL and SFT loop
Launch managed SFT and DPO jobs
Fine-tune top open-source models
Systems research for production AI
All our research publications
Technical docs for Together AI
Our latest news & blog posts
Our open-source demo apps
Practical implementation guides
Build better with open models
Run open models in your harness
Explore our events calendar
Open-source ROI calculator
High-performance inference as APIs
Inference for batch workloads
Token-based capacity with SLAs
Dedicated Model Inference
Inference on custom hardware
Dedicated Container Inference
Inference for custom models
Run open models in your harness
Inference that gets better over time
Explore Top Open-Source Models
Reliable GPU clusters at scale
Custom infrastructure at frontier scale
Build development environments for AI
Store model weights & data securely
Define your own RL and SFT loop
Launch managed SFT and DPO jobs
Fine-tune top open-source models
Systems research for production AI
All our research publications
Technical docs for Together AI
Our latest news & blog posts
Our open-source demo apps
Practical implementation guides
Build better with open models
Run open models in your harness
Explore our events calendar
Open-source ROI calculator
Introducing Together Storage Accelerator: fast, free data caching for your GPU cluster
A distributed cache on every GPU node: about 20x faster warm reads, one origin fetch per cluster, and no application changes. Included free with Together GPU Clusters.
Your GPUs are only as fast as your data pipelines. Every training run that starts with a 30-minute model download, every node that re-pulls the same dataset from across the country, and every Hugging Face rate limit that stalls a job leads to compute resources sitting idle, waiting for your data, while you are paying for them.
Today we're launching Together Storage Accelerator (TSA), a distributed caching layer for Together GPU Clusters that alleviates the need for repeated data reads from object storage and Hugging Face Hub, dramatically accelerating your AI data pipelines while also cutting your data egress bills. TSA is now included free with every Together GPU Cluster.
The problem: your data lives far from your GPUs
Most AI teams keep their source of truth in object storage (Amazon S3, Cloudflare R2, Together AI's T4*) or pull models directly from Hugging Face Hub. That means every cold start pays a WAN tax: cross-cloud or cross-region reads are slow, expensive (egress from AWS runs up to $0.09/GB), and rate-limited. Multiplied across hundreds of nodes pulling the same checkpoint, the waste compounds and costs you time, money, and failed jobs when the origin throttles you.
*T4 is Together AI's in data center S3 compatible object storage solution, currently in Limited Preview. Please contact sales or your account team if you are interested to try it out.
TSA caches the data, improving performance, reliability and cost
TSA provides you a distributed caching layer for GPU clusters. It runs on every GPU node in your cluster and presents drop-in endpoints for the storage you already use. The first time any node reads an object, TSA caches it on cluster NVMe. As a result, every subsequent read, from any node, is served from inside the cluster at dramatically higher local speed.
The result, measured on a Together GPU Cluster H100 node:
~20× faster warm reads than pulling from a cross-cloud origin, at up to 3.4 GB/s per node. A 2 GiB object that took 13-15 s to download from Cloudflare R2 can be read from the cache in 0.6 - 0.7 s
Cold reads cost nothing extra: the first read through TSA ran within ~5% of a direct download
One origin fetch, per cluster. A 70 GB checkpoint pulled by 200 nodes downloads from the source once, not 200 times
Resilience to outages and rate limits. Cached data keeps serving even when the upstream won't.
You can start using TSA without making any changes to your application. No new SDK, no rewritten data loaders, and you can get started with a single command.
# S3-compatible storage (S3, R2, T4, ...)export AWS_ENDPOINT_URL=http://tsa.tsa.svc.cluster.local:18080
# Hugging Face Hubexport HF_ENDPOINT=http://tsa.tsa.svc.cluster.local:18080
Your existing boto3 code, transformers calls, and CLI workflows work as-is. TSA handles authentication with credentials you configure. Credentials are encrypted and securely stored in a secrets vault.
Measured on 2026-10-28 from a pod on one H100 node in a Together GPU Cluster (us-central). Single-stream reads unless noted; clients wrote to /dev/null or node NVMe.
Read a 2 GiB object from a cross-cloud bucket (Cloudflare R2): 143–169 MB/s (12.7–15.0 s) without TSA, 3.0–3.4 GB/s (0.64–0.72 s) with TSA warm. ~20×
Read small files from the same bucket (5 KB, 2.4 MB): 320 ms and 680 ms without TSA, 5 ms and 8 ms with TSA warm. ~60–80× lower latency
200-node cluster pulling the same checkpoint: 200 origin fetches without TSA, 1 with TSA. 200× less egress
Measured on a Together GPU Clusters preview deployment (TSA chart 0.6.0); your results depend on origin, object sizes, and network path. Origin speed is regional: the same Hugging Face download ran 5× faster from us-central than from ca-west, so TSA's win is largest where the origin is slowest. Methodology: single-stream curl and aws s3 cp from a pod on the GPU node, repeated reads of the same object, MD5-verified against direct download.
TSA leverages ASTRA (Accelerated Storage Transport & Retrieval Architecture), the distributed caching technology we built to feed some of the largest GPU fleets in the world, including our own inference fleet. We'll share a deeper technical look at ASTRA in an upcoming engineering post.
TSA works with any S3-compatible store, and it pairs naturally with T4, Together AI's managed object storage solution. Point TSA at your existing S3 or R2 buckets today without migrating your data. When you want your warm data living next to your GPUs permanently, T4 is one config change away. T4 is currently in limited preview: contact sales or your account team for access.
Together Storage Accelerator is available now on Together GPU Clusters at no additional cost. Enable it from your cluster's Settings page, or see the docs to go from zero to accelerated in under ten minutes.
Introducing Together Storage Accelerator: fast, free data caching for your GPU cluster
Auto repair for Together GPU Clusters: from GPU fault to healthy node
Introducing preemptible compute: the same compute, half the price
Together AI and Y Combinator partner to launch the first dedicated GPU cluster for the YC community
New in Together GPU Clusters: Reliability and control for production GPU clusters
Capacity without conflict: A guide to multi-tenant GPU cluster design for AI-native teams
New in Together GPU Clusters: Autoscaling, observability, and self-healing
Inside multi-node training: How to scale model training across GPU clusters
Announcing General Availability of Together Instant Clusters, offering ready to use, self-service NVIDIA GPUs
Bringing 100,000 GPUs to Europe
Together AI Powers Pioneers at GTC: NVIDIA Blackwell GPUs, Instant GPU Clusters, and A Full-Stack for AI Innovation
Introducing Together Instant GPU Clusters Accelerated by NVIDIA GPUs, with Self-Service Provisioning in Minutes
Supercharging NVIDIA H200 and H100 GPU Cluster Performance With Together Kernel Collection
A practitioner's guide to testing and running large GPU clusters for training generative AI models
Growing to 20 exaflops, Together GPU Clusters help startups and enterprises accelerate generative AI development
Optimizing Training Workloads for GPU Clusters
Introducing Together Storage Accelerator: fast, free data caching for your GPU cluster
Last year we built megakernels. This year we deleted them: The road to Together Compiler
Making open models work for long-horizon agentic workloads
How we do agent evals at Together AI
Together Compiler: Continual Compilation for AI inference
From Static to Polymorphic Inference: A new era for production open-source inference systems
Auto repair for Together GPU Clusters: from GPU fault to healthy node
Together Post-Training: frontier-grade model shaping, the way that fits you
Meta Muse Spark Open is live on Together AI
Expanding our enterprise inference capacity with IBM Cloud and NVIDIA
Together Link: frontier-quality open models in the harness you already use
How to train your own Jev for $17
原文为英文,正文由自动抓取并保留原文。
原文发布于 Together AI Blog(Introducing Together Storage Accelerator: fast, free data caching for your GPU cluster),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。