11-19 更新 实时
首页/ 深度/ 正文

别再数 Token,该数真正交付的东西

深度 来源:MIT Technology Review 2026-11-02 21:10 0 阅读 机器翻译
别再数 Token,该数真正交付的东西

当开发者把一项编程任务交给 AI 智能体时,智能体并不只是回答一次。它会规划、生成代码、运行测试,并重复这一过程以修正错误。单个任务消耗的 Token 可能是基础聊天提示词的 10 到 50 倍。智能体越有用,发起的调用就越多。在按 Token 计价的模式下,创造最多价值的工作流也可能变得最为昂贵。

财务压力已经显现。据报道,Uber 在扩大 Claude Code 的使用后,到 4 月就已用尽其 2026 年全部 AI 预算。GitHub Copilot 从固定费率定价转向按量计费的额度。Gartner 预测,到 2028 年,AI 编程的 Token 成本将超过开发者的平均薪资。

自然的应对方式是配给:使用上限、席位限制,以及开发者启动智能体前需获得审批。但一刀切的限制可能会束缚那些正在创造价值的工作流。它们优化的也是错误的指标。每 Token 成本衡量的是消耗,而不是成果。有些支出反映的是浪费,包括重试和失控的循环;有些则代表着交付软件的工作本身。计量表无法区分二者,支出上限同样不能。

更好的衡量指标是每个已合并 pull request 的成本:AI 辅助开发总成本除以通过评审并合并的变更数量。同时跟踪返工与缺陷,以免更便宜的变更掩盖了更低的质量。

如果在不损害质量的前提下,这一成本随着 AI 使用量的上升而下降,说明开发者正在把更多的消耗转化为有用的软件。问题不在于他们用了多少 Token,而在于这笔投入产出了多少完成的工作。

改善这一等式需要两项改变:把昂贵的推理留给真正需要它的任务,并降低常规工作的成本。

智能体式编程并不要求在每一步都使用最强大的模型。对于困难的规划和新颖推理,应使用前沿模型,或在政策要求时使用本地部署的旗舰模型。将适合的工作——包括代码生成、测试创建和反复重构——路由到你自己运营的开放权重模型上。

网关可以根据任务复杂度、数据敏感性、成本和延迟来路由请求。这样就能把高端推理留给真正值得使用它的时刻,而不是把高端定价施加到整个工作流上。为例外情况计量,而不是为整个工作负载计量。

开放权重模型让这种分工越来越可行。软件工程基准测试显示,领先的开放权重模型在某些任务上可以接近前沿性能。但这并不能证明它们在每个代码仓库或工作流中都等效。在分配生产工作之前,要针对你自己的代码、测试和验收标准来评估候选模型。

接下来,决定谁拥有算力容量。租用推理时,每次调用都会增加账单。而拥有自有容量时,大部分成本需预先投入,因此随着使用量增长,每一次额外调用都会变得更便宜。

这笔账很简单:按 Token 计价意味着用量越大越贵,而自有容量会随着团队使用增多而变得更高效。不过,拥有容量并非没有成本。电力、运营、维护和折旧都必须考虑在内。真正关键的指标是每个已合并拉取请求的全成本。

自有算力还能带来一种永远不会出现在账单上的价值:对敏感代码的掌控。当代码生成和测试在本地运行时,专有代码留在企业内部。只有经过批准的规划提示词和受管控的例外情况,才会通过企业自己控制的网关发送给外部模型。这打开了那些因安全顾虑而通常阻碍采用、同时也是最大收益所在之处的敏感代码库。

在 HPE,一个混合 AI 平台将来自 150 多个内部应用的日常负载路由到本地运行的开放权重模型,包括运行在 HPE Private Cloud AI 上的模型。结果是:Token 成本下降 30% 到 50%。

这正是混合模式的意义所在。借助 HPE Private Cloud AI,自持推理并不意味着放弃前沿模型。把大批量负载跑在自有算力上,让网关把最难的任务发出去。

数据还显示,推动支出的是使用量,而非授权人数。前 5% 的用户贡献了大部分消耗,而在同一授权下,使用最重的用户大约是使用最轻用户的 450 倍。如果重度用户从少数人扩大到半个员工队伍,在人数不变的情况下,月度支出可能增长约 4.4 倍。你的构成会有所不同,这正是你需要测量它的原因。

HPE 追踪两项指标,并建议客户也这样做。第一项是每位活跃开发者的成本,因为对所有授权取平均会掩盖推动支出的重度用户。第二项是每个已合并拉取请求的成本,因为它把 AI 支出与真正交付的软件联系起来。

在改变基础设施之前,先为这两项成本建立基线。然后在自有算力上跑一个完整的开发周期。如果随着使用量增长,这两项指标都没有改善,那么你目前的模式可能仍是更好的选择。

自有算力并非适合所有企业。使用量低或难以预测时,更适合采用 API,无需把资金沉淀在闲置的基础设施上。当持续的需求让算力保持忙碌时,自建才划算。大多数企业将采用混合模式:高要求或间歇性的工作负载走前沿模型 API,可重复的大批量负载跑在自有算力上。由受管控的网关决定哪些数据可以离开公司,这样既能用上前沿模型,又不必放弃对自有代码的控制权。

随着开发者越来越依赖编码智能体,使用量会持续上升。真正的选择在于:这种增长是体现为更大的可变成本账单,还是体现为用自有算力交付更多软件。在 AI 辅助开发上领先的公司,不会为了控制成本而限制使用。它们会构建一种运营模式,让 AI 用得越多,经济效益越好,而不是被侵蚀。

如需了解更多关于 HPE Private Cloud AI 的信息,请访问 HPE。

本内容由 HPE 制作,并非由 MIT Technology Review 编辑团队撰写。

译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Stop counting tokens, start counting what ships)

When a developer gives an AI agent a coding task, the agent does more than answer once. It plans, generates code, runs tests, and repeats the process to correct errors. A single task can use 10 to 50 times more tokens than a basic chat prompt. The more useful the agent becomes, the more calls it makes. With per-token pricing, the workflows creating the most value can also become the most expensive.

The financial pressure is already visible. Uber reportedly exhausted its entire 2026 AI budget by April after expanding its use of Claude Code. GitHub Copilot moved from flat-rate pricing to metered credits. Gartner predicts that AI coding token costs will surpass average developer salaries by 2028.

The natural response is rationing usage caps, seat limits, and approvals before developers launch agents. But blanket limits can restrict the workflows delivering value. They also optimize the wrong number. Cost per token measures consumption, not results. Some spending reflects waste, including retries and runaway loops. Some represents the work of shipping software. A meter cannot distinguish them, and neither can a spending cap.

Measure the software that ships

A better measure is cost per merged pull request: total AI-assisted development cost divided by the number of changes that pass review and are merged. Track rework and defects alongside it so cheaper changes do not conceal lower quality.

If that cost falls as AI usage rises, without compromising quality, developers are converting more consumption into useful software. The question is not how many tokens they use, but how much finished work the investment produces.

Improving that equation requires two changes: reserve expensive inference for tasks that need it, and lower the cost of routine work.

Match the model to the task

Agentic coding does not require the most powerful model at every step. Use a frontier model, or an on-premises flagship when policy requires it, for difficult planning and novel reasoning. Route suitable work, including code generation, test creation, and repeated refactoring, to open-weight models you operate yourself.

A gateway can route requests according to task complexity, data sensitivity, cost, and latency. This reserves premium inference for the moments that justify it rather than applying premium pricing to an entire workflow. Meter the exception, not the workload.

Open-weight models make this division increasingly practical. Software-engineering benchmarks show that leading open-weight models can approach frontier performance on some tasks. That does not establish equivalence across every repository or workflow. Evaluate candidates against your own code, tests, and acceptance criteria before assigning production work.

Change the economics of repeated use

Next, decide who owns the capacity. With rented inference, every call adds to the bill. With owned capacity, most of the cost is committed upfront, so each additional call gets cheaper as usage grows.

The economics are simple: per-token pricing makes heavy use more expensive, while owned capacity becomes more efficient as teams use it more. Ownership is not free, though. Power, operations, maintenance, and depreciation all must be considered. The number that matters is the fully loaded cost per merged pull request.

Owned capacity also delivers a type of value that never appears on an invoice: control over sensitive code. When code generation and testing run on-premises, proprietary code stays inside the enterprise. Only approved planning prompts and governed exceptions go to external models, through a gateway the organization controls. That opens the sensitive repositories where security usually stalls adoption, and where the biggest gains are waiting.

Test the operating model

At HPE, a hybrid AI platform routes routine workloads from more than 150 internal applications to open-weight models running on premises, including on HPE Private Cloud AI. The result: token costs down 30 to 50%.

That is the point of hybrid. With HPE Private Cloud AI, owning your inference does not mean giving up frontier models. Run the volume on capacity you own and let a gateway send the hardest work.

Data also shows that usage, not licensed headcount, drives spend. The top 5% of users generated most of the consumption, and the heaviest used roughly 450 times more than the lightest on the same license. If power users grow from a small minority to half the workforce, monthly spend can rise about 4.4 times with flat headcount. Your mix will differ, which is exactly why you need to measure it.

HPE tracks two metrics and recommends customers do the same. First, it tracks cost per active developer, because averaging across all licenses hides the power users driving spend. Second, it tracks cost per merged pull request, because it connects AI spending to software that ships.

Baseline both costs before changing infrastructure. Then run a complete development cycle on owned capacity. If both metrics do not improve as usage grows, your current model may still be the better choice.

Build for adoption, not rationing

Owned capacity is not for everyone. Low or unpredictable usage favors APIs, with no capital tied up in idle infrastructure. Ownership pays off when sustained demand keeps it busy. Most enterprises will run hybrid: frontier APIs for demanding or intermittent work, owned capacity for repeatable volume. A governed gateway decides what leaves the building, so you get frontier models without giving up sovereignty over your code.

As developers lean harder on coding agents, usage will keep rising. The real choice is whether that growth shows up as a bigger variable-cost bill or as more software shipped on capacity you own. The companies that lead AI-assisted development will not ration adoption to control costs. They will build an operating model where more AI use improves the economics instead of undermining them.

To learn more about HPE Private Cloud AI, please visit HPE.

This content was produced by HPE. It was not written by MIT Technology Review’s editorial staff.

正文保留英文原文,标题与摘要由机器翻译。

本文由采集器自动同步

原文发布于 MIT Technology Review(Stop counting tokens, start counting what ships),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗