11-19 更新 实时
首页/ 技术栈/ 正文

Together 后训练:前沿级模型塑造,按你的方式来

技术栈 来源:Together AI Blog 2026-10-19 08:00 0 阅读 机器翻译
Together 后训练:前沿级模型塑造,按你的方式来

推出 Polymorphic Inference,先从 Together Compiler 开始 →

推出 Together Post-Training:模型更好、运行更快、成本更低 →

推出 Together Link:在你已使用的 harness 中运行开放模型 →

开源 AI 技术栈:模型、推理、路由器、harness 与工具 →

GLM-5.3 与 GLM 5.3 Flash 在 DeepSWE 上的对比 →

⚡ 按需 B200 现已在 Together GPU 集群上线 →

以 API 形式提供的高性能推理

面向批处理工作负载的推理

基于 Token 的容量,附带 SLA

专用模型推理

在自定义硬件上进行推理

专用容器推理

针对自定义模型的推理

在您的测试框架中运行开放模型

随时间不断改进的推理

探索顶级开源模型

大规模可靠 GPU 集群

前沿规模的定制基础设施

为 AI 构建开发环境

安全存储模型权重与数据

定义你自己的 RL 与 SFT 循环

启动托管式 SFT 与 DPO 任务

微调顶级开源模型

面向生产级 AI 的系统研究

我们所有的研究出版物

Together AI 的技术文档

我们的最新新闻与博客文章

我们的开源演示应用

实用实施指南

借助开放模型构建更优方案

在你的测试框架中运行开放模型

查看我们的活动日历

开源 ROI 计算器

以 API 形式提供的高性能推理

面向批处理工作负载的推理

基于 Token 的容量,并提供 SLA

专属模型推理

在定制硬件上推理

专属容器推理

面向自定义模型的推理

在自有运行框架中运行开放模型

随时间不断优化的推理

探索顶级开源模型

大规模可靠 GPU 集群

前沿规模的自定义基础设施

为 AI 构建开发环境

安全存储模型权重与数据

定义自己的 RL 和 SFT 循环

启动托管的 SFT 和 DPO 任务

微调顶尖开源模型

面向生产级 AI 的系统研究

我们的全部研究论文

Together AI 技术文档

我们的最新新闻与博客文章

我们的开源演示应用

实用实施指南

用开放模型,构建更优

在你的运行框架中运行开放模型

浏览我们的活动日历

开源 ROI 计算器

Together Post-Training:前沿级模型塑造,以适合你的方式

我们推出 Together Post-Training,这是一项在我们平台上构建自定义模型的新服务——既可通过 RL 从环境奖励中学习,也可通过 SFT 利用监督数据。该服务旨在让你灵活、高效地塑造模型,而无需承担沉重的基础设施负担。

Together Post-Training 让你能轻松定义训练循环,而我们会协同优化底层的系统与基础设施,以提升模型质量和训练速度。现在,你与那个独一无二、专属于你的最佳模型之间再无阻隔。

自定义模型的价值

到 2026 年,为你的 AI 工作负载训练定制模型已不再是小众话题。企业正积极投身其中,以降低成本并提升特定任务上的表现。与其通过上下文工程或配套框架工程来修正模型行为、提升其效率,许多 AI 原生和 AI 前沿组织正在训练专用智能体,以极低的成本与运行时开销达到前沿模型的质量水平。除此之外,自己训练模型还能带来更强的自主权与灵活性:训练出的检查点可以部署在任何位置,可进一步精调,甚至可在本地使用。

在 Together,我们希望让每一位 AI 开发者都能尽可能便捷地获得最适合其任务的模型。对优秀模型进行后训练,需要在数据整理、训练算法和评估协议上反复迭代。此外,强化学习(RL)的基础设施正变得日益复杂,“效率优化”与“拖慢训练运行的隐蔽 Bug”之间往往只有一线之隔。要以可靠且规模化的方式对大模型完成这些工作,本身也带来一系列挑战。

前沿实验室都有完整的团队分别解决上述每一个问题。由于一次性攻克所有挑战意味着巨大投入,许多本可从后训练中受益的企业从未启动,或者启动后中途搁置。Together Post-Training 包揽所有优化与基础设施工作,且不会成为阻碍;我们的前置部署团队还会与客户协作,协助完成端到端流程。这让每位研究者或领域专家都能专注于自己最擅长的事,其余的交给我们。

Together Post-Training 概览

随着后训练的复杂度不断演进,围绕它的工具链也应随之进化。Together Post-Training 提供了一套 ML 开发者所熟悉的通用抽象,同时让你能控制每次实验所用的资源,并确保底层所有计算都快速、低成本,而且最重要的是——正确。

我们构建这项服务,是为了给你的实验提供最大程度的灵活性。你可以运行 LoRA 训练——包括让多个 LoRA 适配器共享同一份 GPU 容量——也可以针对部分模型运行全权重实验。与我们的微调服务不同,Together Post-Training 的整个训练循环完全由你定义。你可以使用最适合自身配置的任意数据、环境或智能体执行框架,并在训练全过程中加入任何你需要的步骤(例如日志记录或外部基准测试)。整个工作流程如下:

使用 Together Post-Training 的高层概览。左侧:在用户侧执行的命令;右侧:在 Together 侧运行的服务组件。

所有实验都运行在 Model Resources 上——这些 GPU 分配完全归你所有,无需与其他请求争抢优先级。每个 Model Resource 可以承载一个或多个 Training Session,每个 Training Session 对应一次使用相同基础模型的实验(例如一组 LoRA 适配器)。这意味着你能获得最佳的 rollout 与训练性能,因为你永远不必与其他用户在同一硬件上执行的请求竞争。同时,专属的 GPU 分配意味着你可以运行全权重微调,在适配知识密集型的大型任务时超越 LoRA 的能力范围。

一旦分配好模型资源并启动会话,你就可以执行训练循环的每一步。每个会话都包含一个 Trainer 实例,负责执行前向/反向步骤并更新权重,还包含若干 Generator 实例,用于根据策略执行 rollout。针对每个模型,我们都对训练和 rollout 性能做了大量优化,确保所有操作都足够快,并且在各技术栈之间保持数值一致。在 SFT 的场景下,你可以启动一个不含 Generator(0 个 Generator)的模型资源来节省成本。由你决定在什么时机把 trainer 参数发送给 Generator;我们还支持异步 RL,让两个组件在任何时候都能保持高 GPU 利用率,尤其是在长时间的智能体式 rollout 场景中。

我们既支持多种标准损失函数(从交叉熵和 GRPO 到 CISPO 和 DPPO),也允许在本地计算自定义的训练目标,并将其梯度发送给 Trainer 来计算更新。

来自 Together Post-Training 文档的、使用自定义损失函数进行训练的示例

Together Post-Training 上线时提供一系列领先的开放权重模型,包括 Muse Spark Open、GLM 5.3、Kimi K3 和 Qwen 3.8-27B。我们将在未来几周内扩充可选范围,确保你既能用上能力最强的前沿 LLM,也能用上高效的小型模型。

质量、性能与效率

把后训练基础设施的内部实现交给外部团队,可能是个令人望而生畏的决定。我们的团队拥有第一手经验,能够在复杂的机器学习系统中揪出那些无声的 bug——从推动 RL 研究的前沿,到训练已服务数百万用户的 LLM。我们明白,最好的训练技术栈就是那个能帮你得到最好模型的技术栈,而构建这样的基础设施,需要在所有组件上进行细致的实现与协同设计。

出于这些原因,我们在开发该服务时,将输出模型的质量作为我们的北极星指标,予以优先考虑。影响 RL 结果的一个典型问题案例是训练引擎与推理引擎之间的对齐:如果相同的模型权重由于数值不匹配而产生略微不同的输出,整个运行过程可能会表现不佳,甚至崩溃。我们消除了训练与推理之间众多导致策略漂移的来源,既采用了 Rollout Routing Replay 等标准技术,也运用了由我们在训练稳定性方面的研究驱动的先进优化手段。

原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Together Post-Training: frontier-grade model shaping, the way that fits you)

Introducing Polymorphic Inference, starting with Together Compiler →

Introducing Together Post-Training: Better models, faster runs, lower costs →

Introducing Together Link: Run open models inside the harness you already use →

The Open Source AI Stack: models, inference, routers, harnesses & tools →

GLM-5.3 vs. GLM 5.3 Flash on DeepSWE →

⚡ On-demand B200s now available on Together GPU Clusters →

High-performance inference as APIs

Inference for batch workloads

Token-based capacity with SLAs

Dedicated Model Inference

Inference on custom hardware

Dedicated Container Inference

Inference for custom models

Run open models in your harness

Inference that gets better over time

Explore Top Open-Source Models

Reliable GPU clusters at scale

Custom infrastructure at frontier scale

Build development environments for AI

Store model weights & data securely

Define your own RL and SFT loop

Launch managed SFT and DPO jobs

Fine-tune top open-source models

Systems research for production AI

All our research publications

Technical docs for Together AI

Our latest news & blog posts

Our open-source demo apps

Practical implementation guides

Build better with open models

Run open models in your harness

Explore our events calendar

Open-source ROI calculator

High-performance inference as APIs

Inference for batch workloads

Token-based capacity with SLAs

Dedicated Model Inference

Inference on custom hardware

Dedicated Container Inference

Inference for custom models

Run open models in your harness

Inference that gets better over time

Explore Top Open-Source Models

Reliable GPU clusters at scale

Custom infrastructure at frontier scale

Build development environments for AI

Store model weights & data securely

Define your own RL and SFT loop

Launch managed SFT and DPO jobs

Fine-tune top open-source models

Systems research for production AI

All our research publications

Technical docs for Together AI

Our latest news & blog posts

Our open-source demo apps

Practical implementation guides

Build better with open models

Run open models in your harness

Explore our events calendar

Open-source ROI calculator

Together Post-Training: frontier-grade model shaping, the way that fits you

We are introducing Together Post-Training, a new service for building custom models on our platform — both from environment rewards via RL and supervision data via SFT. This service is designed to give you flexible and efficient model shaping without the heavy infra burden.

Together Post-Training makes it easy to define the training loop, while we co-optimize the systems and infrastructure underneath for model quality and training speed. Nothing now stands between you and the best model that is uniquely yours.

The value of custom models

In 2026, training custom models for your AI workloads is no longer a niche topic. Companies are actively exploring it to reduce costs and improve task-specific performance. Instead of fixing model behaviors and improving their efficiency via context or harness engineering, many AI-native and AI-forward organizations are training specialist agents, matching the quality of frontier models at a fraction of the cost and runtime. On top of that, training the model yourself yields greater ownership and flexibility: the resulting checkpoint can be deployed in any location, further refined, or even used locally.

At Together, we want to empower every AI developer with the ability to get the best model for their task, as conveniently as possible. Post-training a great model requires many iterations on data curation, training algorithms, and evaluation protocols. On top of that, infrastructure for RL is getting ever more complex, with a fine line between “an efficiency optimization” and “a silent bug that holds back your runs”. Doing so for large models reliably and at scale presents its own set of challenges.

Frontier labs have entire teams solving each of the above problems. Since tackling all challenges at once is a big investment, many companies that could benefit from post-training never start it, or start and stall. Together Post-Training handles all optimizations and infrastructure without getting in the way, and our forward-deployed team partners with our customers to help with the end-to-end workflow. This lets every researcher or domain expert focus on what they do best while we take care of the rest.

Together Post-Training at a glance

As the complexity of post-training continues to evolve, so should the tooling around it. Together Post-Training provides a set of common abstractions familiar to ML developers, as well as control over the resources used by each experiment, while making sure that all computations under the hood are fast, cost-efficient, and, most importantly, correct.

We built our service to offer the highest degree of flexibility for your experiments. You can run LoRA training, including multiple LoRA adapters sharing the same GPU capacity, as well as full-weight experiments for select models. In contrast to our fine-tuning service, the entire training loop of Together Post-Training is yours to define. You can use any data, environments, or agentic harnesses that suit your setup best, and add any steps you need (such as logging or external benchmarking) throughout the training process. The entire workflow is as follows:

A high-level summary of using Together Post-Training. Left: commands executed on the user side, right: service components running on Together’s side.

All experiments run on Model Resources — GPU allocations that are fully yours, not competing with others’ requests for priority. Each Model Resource can hold one or several Training Sessions, where each one corresponds to a single experiment using the same base model (for example, one set of LoRA adapters). This means you get the best rollout and training performance, as you never have to compete with other users’ requests executing on the same hardware. At the same time, dedicated GPU allocations mean you can run full-weight finetuning, going beyond LoRA when adapting to large knowledge-intensive tasks.

Once you allocate the model resource and start the session, you can execute each step of the training loop. Each session contains a Trainer instance responsible for running forward/backward steps and updating the weights, as well as Generator instances for executing rollouts from the policy. For every model, we heavily optimize training and rollout performance, making sure all operations are fast and numerically aligned across stacks. In the case of SFT, you can start a model resource with 0 generators to save costs. You decide the point at which the trainer parameters are sent to the generator; we also support asynchronous RL for high GPU utilization of both components at all times, especially with long agentic rollouts.

While we support a number of standard losses (from cross-entropy and GRPO to CISPO and DPPO), we also allow computing a custom training objective locally and sending its gradients to the Trainer for computing the updates.

Example of training with a custom loss from Together Post-Training docs

Together Post-Training launches with a range of leading open-weight models, including Muse Spark Open, GLM 5.3, Kimi K3, and Qwen 3.8-27B. We will expand the selection in the coming weeks, making sure you can use both the highest-capability frontier LLMs and efficient smaller models.

Quality, performance, and efficiency

Delegating the internals of post-training infrastructure can be a daunting decision. Our team has first-hand experience catching silent bugs in complex machine learning systems, from pushing the frontier of RL research to training LLMs that have served millions of users. We understand that the best training stack is the one that gets you the best model, and building such infrastructure demands careful implementation and co-design across all components.

For these reasons, we prioritized quality of output models as our north star in the development of the service. One notorious example of issues affecting RL results is the alignment between training and inference engines: if the same model weights produce slightly different outputs due to mismatching numerics, the whole run might underperform or even collapse. We eliminate numerous sources of policy drift across training and inference, using both standard techniques like Rollout Routing Replay and advanced optimizations driven by our research in training stability.

Beyond model quality, we also work to make each run as fast and as efficient as possible, because post-training requires multiple iterations: the sooner a run finishes and the less it costs, the more ideas you get to test. Most of the time in RL is spent on generating rollouts, which allows us to capitalize on years of research and engineering behind Together’s inference products. Our rollout engines draw on the same optimizations as our serving stack. For multi-turn agents, intelligent caching and routing minimize redundant computations while keeping memory costs bounded. On the training side, optimized parallelism and sequence packing speed up both SFT and RL steps.

With Together Post-Training, you size each experiment to what it needs and do not pay more. If your experiment is bottlenecked by rollout concurrency, you can request the exact number of sampler replicas. Several LoRA experiments can share one model resource, so running a sweep doesn’t mean reserving more GPUs for every setup. We bill every model resource per GPU-minute, yet thanks to many optimizations of our dedicated and multi-LoRA serving stacks, the overall costs of training drop below token-based pricing as soon as you scale up the experiments.

Optimized deployment after training

When the post-training experiments are finished, it might be important to run each resulting model side by side to compare their outputs. We now support multi-LoRA adapter serving in Dedicated Model Inference: instead of starting a separate production deployment for each run, you can swap the adapters at your convenience in a single GPU deployment, helping you lower the serving costs.

Lastly, converting the model to run at the highest throughput in production means it potentially executes in lower precision, in a different hardware configuration, and with different kernels compared to rollout generation. Through a combination of algorithmic and systems improvements, we ensure that each training checkpoint retains the quality in serving, even after switching from multi-LoRA sampling to a fully performant merged inference in FP4 precision.

To begin your first post-training experiments, install the latest Together SDK and read the docs. We also provide a selection of cookbooks that can help you get started with Together Post-Training, which are available at github.com/togethercomputer/post-training-cookbook. These cookbooks provide examples of training with GRPO on a standard GSM8K benchmark, implementing DPO as a custom loss, or training multiple LoRA adapters within the same model resource. If you already have a setup that uses post-training services such as Tinker API, we also offer a simple way to migrate from them. You can replace one line in your scripts, and everything else can stay the same:

-import tinker +import together.post_training.tinker as tinker

Read our Tinker migration guide to learn more.

How AI-native companies are using Together Post-Training today

Before this launch, we worked with a select group of organizations pushing the frontier of custom models. We granted them early access to our service over the past few months; the feedback we received was invaluable in developing the product which now everyone can use. Hear from the teams themselves:

“We had a great experience with the Together Post-Training platform from day one. The high degree of control and flexibility let us start running real RL experiments within a couple of hours. The service is efficient, allowing us to run up to five concurrent RL experiments on the same GPU capacity. The Together Research team was an exceptional collaborator throughout the training process, giving us feedback on model selection, training techniques, reward functions, and training efficiency.”

– Anthony Li, Associate Director of Data Science, Deel

“At Jack & Jill, we build two AI agents: Jack for professionals, and Jill for companies, who recruit from Jack’s network of professionals who’ve opted in to being introduced. We used Together Post-Training to post-train a model that predicts which candidates a hiring manager will want to meet, a core step in Jill’s search. Onboarding was easy, their team helped design our experiments, and their forward-deployed engineers wrote the scripts that got us to first results. We matched a closed-source model’s quality in offline evaluations, using an open-source model that’s significantly cheaper to serve.”

– Oliver Normand, Founding Engineer, Jack & Jill

“We’ve been collaborating with the Together team since the launch of its Post-Training beta in April. With the latest iteration, seamless self-onboarding made it easy to quickly launch full-scale agentic RL runs. The Post-Training service has proven to be exceptionally intuitive, flexible, and powerful for scaling sophisticated agentic workloads.”

– Adit Jain, Researcher, Collinear AI

Build with Together Post-Training

With the launch of Together Post-Training, we’re turning frontier-grade model customization into a reliable, flexible and efficient service. The next step is for you to make the most of it. For the first eight weeks, we are offering Together Post-Training at a reduced price of $9 per GPU-hour of B200 usage. Learn how to get started today — we’re excited to see what you will build.

But you don’t have to do this alone. For companies that want a partner, our forward-deployed engineers work alongside your team across the full post-training lifecycle. They’ll start with helping select the right open model and agent harness for your use case, work with you on your data and training environments, and then help with evaluations and shipping your own optimized endpoint.

If you would like to collaborate with Together on training models for your organization, feel free to reach out to us.

Together Post-Training: frontier-grade model shaping, the way that fits you

How to train your own Jev for $17

Together AI expands fine-tuning service with more models, live metrics, and finer controls

Announcing Together AI and Adaption Partnership

Together AI expands fine-tuning service with tool calling, reasoning, and vision support

Fine-tuning open LLM judges to outperform GPT-5.2

Together Evaluations now supports comparing top commercial APIs vs. open source models

Fine-Tuning Platform Upgrades: Larger Models, Longer Contexts, Enhanced Hugging Face Integrations

Transform OpenAI gpt-oss Models into Domain Experts with Together AI Fine-Tuning

Fine-Tuning Small Open-Source LLMs to Outperform Large Closed-Source Models by 60% on Specialized Tasks

Together Fine-Tuning Platform, Now With Preference Optimization and Continued Training

Fine-tuning API: Introducing long-context training, conversation data support and more configuration options

Fine-tuning Llama-3 to get 90% of GPT-4’s performance at a fraction of the cost

How to fine-tune an LLM: a developer's guide

RAG vs. fine-tuning: a decision framework for LLM apps

Together Compiler: Continual Compilation for AI inference

From Static to Polymorphic Inference: A new era for production open-source inference systems

Auto repair for Together GPU Clusters: from GPU fault to healthy node

Together Post-Training: frontier-grade model shaping, the way that fits you

Meta Muse Spark Open is live on Together AI

Expanding our enterprise inference capacity with IBM Cloud and NVIDIA

Together Link: frontier-quality open models in the harness you already use

How to train your own Jev for $17

Canary rollouts: upgrade models in production without downtime

How a global fintech scaled coding agent traffic with Dedicated Model Inference

Migrating from closed to open source models, Together

Together AI expands fine-tuning service with more models, live metrics, and finer controls

原文为英文,正文由自动抓取并保留原文。

本文由采集器自动同步

原文发布于 Together AI Blog(Together Post-Training: frontier-grade model shaping, the way that fits you),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗