11-19 更新 实时
首页/ 技术栈/ 正文

让开放模型胜任长周期智能体任务

技术栈 来源:Together AI Blog 2026-11-16 08:00 0 阅读 机器翻译
让开放模型胜任长周期智能体任务

推出 Polymorphic Inference,首先从 Together Compiler 开始 →

推出 Together Post-Training:模型更优、运行更快、成本更低 →

推出 Together Link:在你已在使用的 harness 中运行开放模型 →

开源 AI 技术栈:模型、推理、路由、harness 与工具 →

GLM-5.3 与 GLM 5.3 Flash 在 DeepSWE 上的对比 →

⚡ 按需 B200 现已在 Together GPU 集群上线 →

以 API 形式提供的高性能推理

面向批处理工作负载的推理

基于 Token 的容量,提供 SLA

专用模型推理

在自定义硬件上推理

专用容器推理

面向自定义模型的推理

在您的执行框架中运行开放模型

随时间不断优化的推理

探索顶级开源模型

大规模可靠的 GPU 集群

前沿规模的自定义基础设施

为 AI 构建开发环境

安全存储模型权重与数据

定义你自己的 RL 与 SFT 循环

启动托管的 SFT 与 DPO 任务

微调顶级开源模型

面向生产级 AI 的系统研究

我们所有的研究出版物

Together AI 的技术文档

我们的最新新闻与博客文章

我们的开源演示应用

实用实施指南

用开放模型构建更好的应用

在你的测试框架中运行开放模型

浏览我们的活动日历

开源 ROI 计算器

以 API 形式提供的高性能推理

面向批处理工作负载的推理

按 Token 计量的容量,附带 SLA

专属模型推理

基于定制硬件的推理

专属容器推理

面向自定义模型的推理

在你的运行框架中运行开放模型

随时间不断变好的推理

探索顶级开源模型

大规模可靠 GPU 集群

前沿规模的自定义基础设施

为 AI 构建开发环境

安全存储模型权重与数据

定义你自己的 RL 与 SFT 循环

启动托管式 SFT 和 DPO 任务

微调顶尖开源模型

面向生产级 AI 的系统研究

我们的全部研究论文

Together AI 技术文档

我们的最新新闻与博客文章

我们的开源演示应用

实用实施指南

用开放模型构建更好的应用

在你的 harness 中运行开放模型

浏览我们的活动日历

开源 ROI 计算器

让开放模型胜任长周期智能体式工作负载

harness 和推理引擎各自必须做对什么,才能让长时间运行的智能体会话在面对长周期任务时保持高速。

编码智能体在每一轮都会把整段对话重新发送一遍。前缀缓存可以让推理引擎不必重做这部分工作,但前提是 harness 不去改动提示词的开头部分,并且引擎要把每一轮都送回已经缓存了该前缀的那块 GPU。Together AI 和 Ollama 处在这一交接的两端,因此我们记录下客户端侧和推理引擎侧各自需要做什么,才能成功构建智能体式应用。

编码智能体的流量模式很奇特。智能体每走一步,harness 都会把到目前为止的整段对话发给模型:系统提示词、每一个工具定义、你的请求、智能体读过的每个文件,以及它运行过的每条命令。而返回的内容通常只是一个几十个 Token 长的工具调用。接着同样的过程再来一遍,带上上一轮的全部内容,再外加一点新东西。

这种输入序列长度(ISL)大、输出序列长度(OSL)小的特征,是大多数智能体式工作负载的典型形态。如果 GPU 每一轮都要从头处理完整个对话记录,长会话就会随着每一步推进而变得更慢、更贵。前缀缓存挽救了其中大部分重复计算。但问题在于,稍有不慎,前缀缓存也很容易被破坏。

我们从两个相反的方向切入这个问题。Ollama 负责的是 harness 集成,也就是决定每个请求里放什么内容。Together AI 负责的是推理引擎,也就是决定请求到达后如何处理。我们不断发现,双方各自的很多工作,其实都是在保护缓存不被对方的某些行为破坏,所以我们想把这些内容集中写下来,帮助智能体开发者和基础设施开发者。

从 GPU 的视角看,一轮智能体交互是什么样的

每个请求都要经过两个阶段。第一阶段是预填充:引擎通过一次大规模并行计算读完整个提示词,并生成第一个 Token。GPU 擅长做这件事,因为这相当于一次性完成大量数学运算,能把算力强度拉到最高。预填充决定了你要等多久才会看到第一个新 Token 出现。第二阶段是解码:模型逐个 Token 写出剩余的回答,而每一个新 Token 都需要回读并关注此前的全部内容。解码更多受限于内存带宽,而非算力,它决定了文本以多快的速度流式输出。

普通的聊天响应会在两个阶段都耗时,而智能体的一轮操作主要花在 prefill 阶段,因为输入是整个会话,输出通常只是一次简短的工具调用。推理模型会在此基础上再额外生成思考 Token,有时数量还相当多,但随着会话增长到数万 Token,输入仍然占据主导。每一轮输入都会变得更长,而输出大小基本保持不变——也就是几次工具调用。

因此对智能体来说,“这个模型有多快?”基本等同于“它处理一长串过往轮次的历史记录有多快?”要快速处理带有大量历史轮次的任务,办法就是把它缓存起来直接读回,而不是重新计算。

理解 KV 缓存(甚至前缀缓存也是如此)最直观的方式就是拿阅读作类比。想象一下,为了读出页面上的每一个新词,你都必须从最上面重新开始。你读得越往下,速度就越慢,因为你不得不反复重读已经读过的所有内容。这就是没有缓存的模型。缓存在单次请求内解决了这个问题:推理引擎保留此前对各 Token 所做的计算(以中间 key、value 对的形式存储),只计算新增的那一个 Token。

前缀缓存则在跨请求的场景下做同样的事。智能体会话的第 7 轮就是第 6 轮再加上一点新增内容,因此如果引擎仍然保留着第 6 轮的计算结果,它就只需处理新增的那部分。

算术很简单。如果每一轮都新增相同的量,不做缓存的十轮会处理 1 + 2 + … + 10 = 55 个单位的提示词。有了缓存,它们只处理 10 个。到一百轮时——对编码任务来说并不罕见——就是 5,050 与 100 的对比。实际每一轮的大小各不相同,所以这些只是理想化的数字。缓存会话也并非完全免费,因为每个新 Token 仍然要对已缓存的 Token 做注意力计算。但这个规律依然成立:没有缓存时,你重复做的工作量随会话长度的平方增长。

问题在于,前缀缓存从提示词开头开始匹配,一直匹配到第一个不同的 Token 就停下。第一个差异之后的所有内容都必须重新处理。

在提示词末尾追加内容不会带来额外开销。改动更靠前的任何内容,哪怕只改一个字符,都意味着它之后的所有内容要重新处理。如果改动靠近开头,那实际上等于丢掉了整个会话,现在必须重建缓存。本文其余几乎全部内容都是从这一条规则推出来的。

第一个改动落在什么位置,由 harness 决定。下一个请求到来时缓存是否还在,由引擎决定。我们先从 harness 说起。

在 Ollama 这边,我们推出了通过 ollama launch 命令连接你常用 harness 的能力。这样你就可以把 Claude Code、Codex、Pi(以及更多)这类 harness 与 DeepSeek-V4.1-Flash、GLM-5.3 等开放模型连接起来,同时保持你现有的工作流。

把 harness 连接到 API 看似简单,但 harness 与模型之间存在多层转换,双方的预期也各不相同。这往往是缓存失效(带来时间和金钱成本)、智能体行为无法解释,以及 harness 直接报错的原因。

如今的智能体框架(harness)携带的工具相当多。如果每个定义都预先塞进提示词,你就会把上下文浪费在任务永远不会用到的工具上,模型挑选正确工具的能力也会变差。较新的框架加入了工具搜索功能来缓解这一问题:智能体可以自行搜索到某个工具,然后加载并使用它。

假设你有一个具备该能力的个人助理智能体。如果你让它把你加星标的事件导出到日历,它会搜索 "calendar",加载 save_calendar 的定义,然后调用它。

就缓存而言,问题在于这个定义放在哪里。

最直接的做法是把定义加入工具列表。这可行,但工具列表位于提示词靠前的位置,在对话之前。因此一旦前缀发生变化,其后的所有内容都要重新处理。在长会话中,这可能意味着数十万个 Token。更好的做法是把定义放在末尾,也就是新内容本来就会写入的位置。

只不过并非每个模型都支持这种做法。有些对话模板不期望工具定义出现在对话中途,调用就会报错。所以,框架能否在不付出代价的情况下延迟加载工具,取决于模型。用户不必知道这一点,但任何构建框架的人都必须知道。

原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Making open models work for long-horizon agentic workloads)

Introducing Polymorphic Inference, starting with Together Compiler →

Introducing Together Post-Training: Better models, faster runs, lower costs →

Introducing Together Link: Run open models inside the harness you already use →

The Open Source AI Stack: models, inference, routers, harnesses & tools →

GLM-5.3 vs. GLM 5.3 Flash on DeepSWE →

⚡ On-demand B200s now available on Together GPU Clusters →

High-performance inference as APIs

Inference for batch workloads

Token-based capacity with SLAs

Dedicated Model Inference

Inference on custom hardware

Dedicated Container Inference

Inference for custom models

Run open models in your harness

Inference that gets better over time

Explore Top Open-Source Models

Reliable GPU clusters at scale

Custom infrastructure at frontier scale

Build development environments for AI

Store model weights & data securely

Define your own RL and SFT loop

Launch managed SFT and DPO jobs

Fine-tune top open-source models

Systems research for production AI

All our research publications

Technical docs for Together AI

Our latest news & blog posts

Our open-source demo apps

Practical implementation guides

Build better with open models

Run open models in your harness

Explore our events calendar

Open-source ROI calculator

High-performance inference as APIs

Inference for batch workloads

Token-based capacity with SLAs

Dedicated Model Inference

Inference on custom hardware

Dedicated Container Inference

Inference for custom models

Run open models in your harness

Inference that gets better over time

Explore Top Open-Source Models

Reliable GPU clusters at scale

Custom infrastructure at frontier scale

Build development environments for AI

Store model weights & data securely

Define your own RL and SFT loop

Launch managed SFT and DPO jobs

Fine-tune top open-source models

Systems research for production AI

All our research publications

Technical docs for Together AI

Our latest news & blog posts

Our open-source demo apps

Practical implementation guides

Build better with open models

Run open models in your harness

Explore our events calendar

Open-source ROI calculator

Making open models work for long-horizon agentic workloads

What the harness and the inference engine each have to get right so a long agent session stays fast for long horizon tasks.

Coding agents send the whole conversation again on every turn. Prefix caching keeps the inference engine from redoing that work, but only if the harness leaves the start of the prompt alone and the engine sends each turn back to the GPU that already has it cached. Together AI and Ollama work on opposite sides of that handoff, so we wrote down what each, the client side and the inference engine side, needs to do to successfully build agentic apps.

Coding agents have an odd traffic pattern. Every time the agent takes a step, the harness sends the model the entire conversation so far: the system prompt, every tool definition, your request, every file the agent has read, and every command it has run. What comes back is usually a tool call a few dozen tokens long. Then it happens again, with everything from last time plus a little more.

This large input sequence length (ISL) and small output sequence length (OSL) is characteristic of most agentic workloads. If the GPU had to work through that whole transcript from scratch on every turn, long sessions would get slower and more expensive with every step. Prefix caching is what saves most of that work. The problem is that if you’re not careful it's also very easy to break the prefix cache.

We come at this problem from opposite ends. Ollama works on harness integrations which means deciding what goes into each request. Together AI works on the inference engine which means deciding what happens when that request arrives. We kept finding that much of our work on each side is about protecting the cache from things the other side can do to it, so we wanted to write it all down in one place to help agent and infra builders alike!

What an agent turn looks like to a GPU

Every request goes through two phases. The first is prefill: the engine reads the whole prompt in one big parallel pass and produces the first token. GPUs are good at this because it's a lot of math done all at once which maximizes arithmetic intensity. Prefill determines how long you wait before any new tokens appear. The second is decode: the model writes the rest of the answer one token at a time, and each new token requires reading back and attending through everything before it. Decode is limited by memory bandwidth more than by compute, and it determines how fast the text streams out.

A normal chat response spends time in both phases while an agent turn is mostly prefill, because the input is the whole session and the output is usually a short tool call. Reasoning models add thinking tokens on top of that, sometimes a lot of them, but as a session grows into the tens of thousands of tokens the input still dominates. Each turn the input gets longer, and the output stays roughly the same size — a few tool calls.

So for agents, "how fast is this model?" mostly means "how fast can it get through a long historical account of pervious turns?" The way to process a task with lots of previous turns quickly is to just cache it and read it back in rather then recomputing.

The easiest way to picture the KV cache(or even the prefix cache, for that matter) is reading. Imagine that to read each new word on a page, you had to start again from the top. The further down the page you got, the slower you'd go, because you'd keep having to reread everything you'd already read. That's a model with no cache. The cache solves this within a single request: the engine keeps the work it did on earlier tokens(stored as intermediate key, value pairs) and computes only the new one.

Prefix caching does the same thing across requests. Turn 7 of an agent session is turn 6 plus a bit more, so if the engine still holds turn 6's work, it only has to process the new part.

The arithmetic is simple. If each turn adds the same amount, ten turns without caching process 1 + 2 + … + 10 = 55 units of prompt. With caching, they process 10. At a hundred turns, which isn't unusual for a coding task, it's 5,050 versus 100. Real turns vary in size, so treat these as idealized numbers. A cached session isn't completely free either, since every new token still has to attend to the cached ones. But the pattern holds: without caching, the work you redo grows with the square of the session length.

The catch is that a prefix cache matches from the beginning of the prompt up to the first token that's different, and stops there. Everything after that first difference has to be processed again.

Adding to the end of the prompt costs nothing extra. Changing anything earlier, even a single character, means re-processing everything after it. If the change is near the top, you've effectively thrown away the whole session and now must rebuilt the cache. Almost everything else in this post follows from that one rule.

The harness decides where that first change lands. The engine decides whether the cache is still there when the next request shows up. We'll start with the harness.

On the Ollama side, we shipped the ability to connect your favorite harness through the ollama launch command. So you can connect harnesses like Claude Code, Codex, Pi (and more) with open models like DeepSeek-V4.1-Flash and GLM-5.3 and maintain your existing workflows.

Connecting a harness to an API might seem easy, but there are many layers of translation and varying expectations between a harness and a model. This often is the cause of cache breakages (incurring time and monetary costs), unexplainable agent behavior, and the harness outright erroring.

Harnesses carry quite a few tools now. If every definition goes into the prompt up front, you spend context on tools the task will never use, and the model gets worse at picking the right one. Newer harnesses have added a feature for tool search to alleviate this, where the tool can be loaded and used by the agent itself after searching for it.

Let’s say you have a personal assistant as an agent which has this capability. If you ask the agent to export the events you've starred to a calendar, and it searches for "calendar," loads the save_calendar definition, and calls it.

For the cache, the question is where that definition goes.

The obvious approach is to add the definition to the tool list. That works, but the tool list sits near the top of the prompt, before the conversation. So if the prefix changes everything after it gets processed again. On a long session, that can be hundreds of thousands of tokens. The better approach is to put the definition at the end, where new content goes anyway.

Except not every model supports that. Some chat templates don't expect a tool definition halfway through a conversation, and the call errors out. So whether a harness can load tools late without paying for it depends on the model. Users shouldn't have to know this, but anyone building a harness does.

Things at the top that change

The easiest way to wreck the cache is to put something in the system prompt that changes from turn to turn, like a current timestamp, the git status, or a working directory the agent keeps moving through. If the harness re-renders the system prompt with fresh values on every turn, every turn is a full re-read.

Things that are fixed for a session but differ between sessions, like the user's name or the date the session started, don't hurt that session. They do stop new sessions from reusing a setup that's already cached, which we'll get to on the engine side. Either way, the fix is the same: put that information in the task message, once, and keep the system prompt identical everywhere.

We hit this one on the Ollama side. In GLM-5.3-Flash, reasoning effort isn't a separate setting the engine handles on the side. The chat template writes it into the very first system block, ahead of the tools and the conversation.

Let’s say you're deep into a long task and it isn't going well. You raise effort from low to high. That's a reasonable thing to do, but the next request now differs from the last one almost at the first token. Which means the whole session gets processed again. On a long run, that can be hundreds of thousands of tokens spent on changing one setting.

Our advice is to pick an effort level at the start and leave it there. If you really need more thinking, it's often cheaper to start a fresh session at the higher setting than to continue the old one, and you get a clean context out of it too. You could also compact before changing the reasoning value to decrease the amount of prompt tokens.

A tool’s output becomes part of the next request, and of every request after it. If the agent reads a 3,000-line file when it needed forty lines, you don't pay for that once; you pay for it on every remaining turn (even if it is cached you lose out on the context window). To combat this, harnesses often cut tool output down. That's the right call, but how you cut tool output matters.

Here's a failure that's easy to build by accident. The task is to fix a timezone bug in a calendar export. The agent reads calendar.py, and the tool returns the first 100 lines of a 640-line file without saying it stopped. The function that matters, convert_timezone, is at line 420. The model has no idea it's missing anything.

The fix is a single line in the output, telling the model it got lines 1 to 100 of 640 and that it can search the file or request another range. With that, the model searches for "timezone," finds line 420, and reads lines 410 to 450. Cutting output saves money. Cutting it without saying so gets you a model confidently editing code it never saw.

Open models like Kimi K3, GLM-5.3, and GLM-5.3-Flash are now pretty good at splitting a big task and handing pieces to subagents. How the harness starts each subagent decides how much of the parent's cache it can reuse.

One option is for the subagent to inherit everything: the same system prompt, tools, and conversation, plus its assignment. Side chats work this way. They branch off the parent session, so the whole prefix is already cached. The other option is to send a brief: the same system prompt and tools, then only the files and instructions the subagent needs. In that case the cache covers the setup and nothing more.

Inheriting sounds strictly better, but it isn't. A cached prefix is cheap to process, not free. Every token the subagent generates still has to read through it, and it occupies GPU memory the whole time. A reviewer that needs two files and a checklist probably doesn't need three hundred turns of history. And if you give a subagent its own system prompt, it shares nothing with the parent at all, not even the setup.

You can do everything above perfectly and still not benefit. The cache has to still exist, and the next request has to reach the machine that holds it.

At Together AI, we think of an inference engine as a machine for keeping GPUs busy. A single GPU costs around $50k to rent per year, so ideally it should never sit idle, and at any moment it's serving hundreds of users who are each at a different point in their request. Agent traffic pushes on those constraints in a few specific ways.

Routing requests for optimal cache hits

A session's cache lives on a particular node in a particular data center. If the next turn in a conversation lands anywhere else, that node has to process all earlier turns from scratch, and now two nodes are storing the same prefix. In order to route correctly we First need to pick a data center that runs the model, has capacity, and is where the session has been running. Then we need to pick a node inside it that has the previous turns cache.

There are two main ways to pick the node. The simple one is to use a key: the client sends something like prompt_cache_key with every request and then ideally we route every request with the same key to the same node. The other way is to hash the prefix and send the request to whichever node has the longest matching prefix cached, even if another session put it there.

That second option is the other reason to keep the system prompt identical. If every session of a harness starts with exactly the same system prompt and tool definitions, a brand-new session might be able to find that setup already cached. If you put unique identifiers in there then every session is guaranteed to start cold.

It also isn't as simple as always sending the request to the warm node. Sometimes the warm node is overloaded with traffic, and then the choice is between waiting in line there or taking an idle node and paying to process the prefix again. In this load-aware routing paradigm sometimes the cache miss might be the better deal. So even a perfectly stable prompt doesn't guarantee a hit if the infernce engine on the backend is not implemented correctly.

Keeping the cache alive between turns

Routing only helps if there's still a cache to route to. An agent will often make a tool call and then wait a couple of minutes while tests run. Keeping that session's cache in GPU memory the whole time takes space active users need, and throwing it away means re-reading everything when the agent comes back from its tool calls. So we can also offload the cache to CPU memory while the session is idle and restore it for the next turn.

Memory is the constraint behind most of this. We can't know ahead of time whether a response will be 30 tokens or 30,000, so instead of reserving room for the worst case max context length scenarios, we hand out cache memory in small blocks as each request needs them (PagedAttention), the way an operating system handles RAM. And we rebuild the running batch on every pass (continuous batching), so when an agent fires off five subagents at once, each new request takes a slot the moment one frees up instead of waiting for the slowest one in a batch to finish.

The upshot for anyone building a harness is that headroom for cache memory, not compute, is usually what limits how many requests a GPU can serve at once. Every oversized tool result, and every subagent carrying irrelevant history takes memory that another session's cache could have used.

Agents also push the prefill/decode split to its extreme, with enormous prompts and tiny answers. To better optimize this we run the two phases on separate clusters. One is tuned for compute and does prefill; the other is tuned for memory bandwidth and does decode for hundreds of streams at once, with the cache shipped between them over RDMA. That way, a wave of long agent prompts doesn't slow token streaming for everyone else.

Putting it all together:One session, both sides

Here's what all of this looks like in a single session.

On the harness side, the agent finds the calendar tool and adds it at the end of the conversation, so nothing above it changes. It reads calendar.py, sees that it got only lines 1 to 100 of 640, searches for "timezone," and reads the right section. Effort stays where it started. A reviewer subagent inherits the session to check agenda.ics against sessions.json.

On the engine side, every turn goes back to the node that has the session cached. While the tests run, the cache waits in CPU memory. The reviewer's request slots into the running batch, and each turn processes only what's new.

None of that is visible to the person using the agent; they just see it stay fast at turn ninety. But it only works if both sides do their part. A harness can keep the prompt perfectly stable and still lose the cache if the engine sends turn 7 to the wrong machine. An engine can route perfectly and still lose it if the harness changes the first line every turn.

Parth writes more about building agents at parthsareen.com, and Ollama posts updates at ollama.com/blog. You can also read more about and run the open models mentioned here on https://www.together.ai/blog.

Making open models work for long-horizon agentic workloads

Together Link: frontier-quality open models in the harness you already use

Canary rollouts: upgrade models in production without downtime

How a global fintech scaled coding agent traffic with Dedicated Model Inference

Migrating from closed to open source models, Together

The Open Source AI Stack

A/B test models in production

Autoscaling endpoints for LLM inference

Configuring Dedicated Model Inference

The production platform for open-weight AI inference

What does 99.9% uptime mean for inference?

Together AI brings Thinking Machines Lab’s new model Inkling on day 0

Open, convenient and predictable: Introducing Provisioned Throughput

Kimi K2.7 Code vs Claude Fable 5: Landing pages that cost 94% less

Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets

How Together AI built the world’s fastest speech-to-text stack

Benchmarking inference at scale: coding agents

Together AI and Pearl Research Labs Team Up to Reduce the Cost of AI Inference

Serving DeepSeek-V4: why million-token context is an inference systems problem

Deploy and inference any model from HuggingFace

Foundational research powering efficient inference at scale

Introducing Dedicated Container Inference: Delivering 2.6x faster inference for custom AI models

Optimizing inference speed and costs: Lessons learned from large-scale deployments

Learn how Cursor partnered with Together AI to deliver real-time, low-latency inference at scale

Together AI delivers fastest inference for the top open-source models

Dynamic AI agent testing for the real world with Collinear Simulations and Together Evals

Improved Batch Inference API: Enhanced UI, Expanded Model Support, and 3000× Rate Limit Increase

Together Evaluations: Benchmark Models for Your Tasks

Together AI Delivers Top Speeds for DeepSeek-R1-0528 Inference on NVIDIA Blackwell

Introducing the Together AI Batch API: Process Thousands of LLM Requests at 50% Lower Cost

Introducing Together Code Sandbox & Together Code Interpreter: SOTA code execution for AI

Together Code Sandbox: the most robust infrastructure for building AI coding products at scale

Together Code Interpreter: execute LLM-generated code seamlessly with a simple API call

Introducing Together Chat: use DeepSeek R1 for free, hosted in North America

Deploy Leading AI Models Accelerated by NVIDIA NIM on Together AI

On-demand dedicated endpoints: run inference with unmatched price-performance & control at scale

Deploy DeepSeek-R1 at scale: Fast, secure serverless APIs and large-scale Together Reasoning Clusters

Generate images with specific styles using Flux LoRAs on Together AI

Announcing Together Inference Engine 2.0 with new Turbo and Lite endpoints

Using Axiomic to build multi agent chat with Together API

Announcing v1 of our Python SDK

Announcing function calling and JSON mode

Building your own RAG application using Together AI and MongoDB Atlas

Building your own RAG application using Together AI and Langchain

Building your own RAG application using Together AI and LlamaIndex

Introducing the Together Embeddings endpoint — Higher accuracy, longer context, and lower cost

Filter responses of any model with Llama Guard or your own safety model

Announcing Together Inference Engine – the fastest inference available

Announcing Together Custom Models. Build a state-of-the-art LLM with Together AI — and own the model.

Together AI launches full stack for developers to build with open-source AI

RedPajama 7B now available, instruct model outperforms all open 7B models on HELM benchmarks

RedPajama-INCITE-3B, an LLM for everyone

Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models

RedPajama, a project to create leading open-source models, starts by reproducing LLaMA training dataset of over 1.2 trillion tokens

OpenChatKit now runs on consumer GPUs with a new 7B parameter model

LLM inference: how it works

KV cache explained: how it works and what it costs

What are LLMs? How large language models actually work

How to Build a State-of-the-Art Search Stack for LLMs: RAG, Reranking, and Reinforcement Learning

Best practices to accelerate inference for large-scale production workloads

Making open models work for long-horizon agentic workloads

How we do agent evals at Together AI

Together Compiler: Continual Compilation for AI inference

From Static to Polymorphic Inference: A new era for production open-source inference systems

Auto repair for Together GPU Clusters: from GPU fault to healthy node

Together Post-Training: frontier-grade model shaping, the way that fits you

Meta Muse Spark Open is live on Together AI

Expanding our enterprise inference capacity with IBM Cloud and NVIDIA

Together Link: frontier-quality open models in the harness you already use

How to train your own Jev for $17

Canary rollouts: upgrade models in production without downtime

How a global fintech scaled coding agent traffic with Dedicated Model Inference

原文为英文,正文由自动抓取并保留原文。

本文由采集器自动同步

原文发布于 Together AI Blog(Making open models work for long-horizon agentic workloads),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗