11-19 更新 实时
首页/ 技术栈/ 正文

降低生成式 AI 成本:规则用代码,判断用模型

技术栈 来源:AWS Machine Learning 2026-11-18 05:14 0 阅读 机器翻译
降低生成式 AI 成本:规则用代码,判断用模型

生成式 AI 系统中最大的成本杠杆,在于你是否要调用模型。我们经常看到团队把每项任务都经由大语言模型(LLM)处理,包括输入校验、格式化和路由。单次调用看似便宜,但在生产规模下,这些小额成本会成为架构中最大的支出项之一。只要确定性规则能完成工作,跳过模型就能省钱、降低延迟,并消除一个非确定性来源。本文将介绍如何判断工作流中的每个步骤是否真的需要模型。随后你可以把每种选择映射到 AWS:确定性代码、受约束的决策调用,或在 Amazon Bedrock 上选用规格合适的模型。

这种模式通常始于原型阶段。模型调用在某个步骤上奏效,于是它就成了下一步的默认做法。这些步骤包括校验输入、重新格式化请求体和选择路由。在原型规模的调用量下,没人会注意到成本。

在生产规模下,不必要的模型调用会从三个方面带来成本:

此后每当有新模型发布,关于该用哪个模型的争论就会重新开始。而更早的那个问题——这个步骤到底需不需要模型——却无人问及。

包括 AWS Well-Architected 生成式 AI Lens 在内的成本指导,往往从模型选型开始。本文则提前一步:这个步骤到底需不需要模型?

图 1 以流程图展示了整个决策过程,其中有三条带标注的泳道:以代码运行(Runs as code)、问题(Questions)、在模型上运行(Runs on a model)。让工作流的每个步骤依次经过三个问题,并在第一个满足该步骤要求的选项处停止:

如果这三个问题的答案都是“否”,则该步骤属于开放式任务,应在规格合适的模型上运行。

图 1:判断工作流步骤是否需要模型,以及该用哪个模型

设想一个客户支持助手。下表展示了它的五个步骤是如何走完这张流程图的。

当任务是开放式的、含义模糊的或需要生成内容时,你可以使用模型:例如总结文档、起草回复、从文本中提取含义。当任务存在可校验的规则时,就应使用确定性代码:例如邮箱格式、schema、精确关键词、数值范围。下表展示了当你把一个规则型任务交给模型时,会牺牲掉什么。

模型还可能产生幻觉,并对同一输入返回不同的答案。temperature 设为 0 可以降低答案出现差异的可能性,但并不能保证输出完全一致。Amazon Bedrock 的模型评估和 Amazon Bedrock Guardrails 可帮助你衡量并控制这些风险。规则不存在这两个问题,这也是受监管的工作负载偏好规则的原因之一。一个有用的检验方法:如果你能把验收测试写成一条规则,那么通常也能把该步骤写成一条规则。更多信息请参阅 AWS Prescriptive Guidance 中的《Orchestration models: From rule-based to AI-native》。

大多数路由和分类步骤介于规则与开放式生成之间。规则无法判定所有情况,但这些步骤也不需要生成文本。它们需要的是从你预先定义的固定选项列表中给出一个答案,就像前一个例子中的支持队列一样。应使用一个会把输出限制在允许答案范围内的组件,并在需要升级时给你一个信号。置信度高的答案继续流转,不确定的则升级处理。AWS Well-Architected 的 Agentic AI Lens 描述了这种分层式、基于置信度的升级机制。在 AWS 上,你可以用四种方式来构建它:

下面的 JSON schema 把结构化输出的答案限制为四个队列加上“unsure”。Amazon Bedrock 要求 schema 中的每个对象都必须把 additionalProperties 设为 false。

{ "type": "object", "properties": { "queue": { "type": "string", "enum": ["billing", "technical", "account", "other", "unsure"] } }, "required": ["queue"], "additionalProperties": false } 无论你选择哪个选项,都要用自己标注的数据来校验它的升级信号,并在这里设定阈值。把退款额度之类的授权决策保留在规则里。

许多成本模式都聚焦于模型之间的路由。而快速路径要更早一步:先运行确定性检查,只把规则解决不了的部分交给模型。意图路由就是一个例子。许多消息带有明确无误的信号,比如“refund”或“reset my password”,关键词匹配可在个位数毫秒甚至更短时间内解决。如果规则能解决 95% 的请求,那么只有 1/20 的请求会到达模型——这还没算上模型选择带来的节省。(此为示意性算术,并非基准测试。)

在 AWS 上,你可以用现有的构建模块来组合实现:

下面的 Choice 状态就是一条关键词快速路径。它使用 JSONata 把退款和密码重置消息直接送到代码,把其他所有消息送到受限的决策调用。

"KeywordFastPath": { "Type": "Choice", "QueryLanguage": "JSONata", "Choices": [ { "Condition": "{% $contains($states.input.message, /refund/i) %}", "Next": "RefundFlow" }, { "Condition": "{% $contains($states.input.message, /reset (my )?password/i) %}", "Next": "PasswordResetFlow" } ], "Default": "ConstrainedDecisionCall" } 你也可以让模型把规则写一次,而不是每次都运行它。让它起草一个正则表达式或 JSON schema,审查后提交。运行时,规则以规则本身的速度和成本运行。审查模型编写的正则表达式,并针对已知的合格与不合格输入进行测试。避免使用嵌套量词,例如 (a+)+,这类写法在某些输入上会让匹配变得非常缓慢(即正则表达式拒绝服务,ReDoS)。

当你确实需要生成时,Amazon Bedrock 结构化输出会把响应约束为你定义的 JSON schema。响应随后会匹配该 schema,因此你的代码无需自定义解析逻辑即可解析(仍需检查内容是否正确)。把范围检查(例如数值边界和长度限制)保留在代码中。有关支持的模型和 schema 特性,请参阅 Get validated JSON results from models。

如果你的规则存放在配置中,只允许使用已知的原语,例如正则表达式、schema 检查、关键词列表和范围检查。绝不要从配置文件中运行任意代码。

选择满足你质量下限的最便宜、最快的模型。这通常并不是最新或最大的模型。Generative AI Lens 在其关于合理选择模型规模的最佳实践中涵盖了这一点。《Amazon Bedrock 的有效成本优化策略》对此有深入介绍。在比较模型之前先定义需求:

Amazon Bedrock 在一个 API 背后提供众多模型系列和多种规模,因此你无需改动集成方式,就能为每项任务选配恰当规模的模型。Amazon Bedrock Intelligent Prompt Routing 会把每个请求路由到受支持模型系列中合适的模型。Amazon Bedrock Model Distillation 可将知识从较大的模型迁移到更小、更经济的模型,以适配你的用例。模型提供商会用广泛的基准测试来评估新的前沿模型,但你的工作负载只是单一任务,只有单一质量下限。更大的模型可能增加成本和延迟,却不会改善那项任务。在切换之前,请对照你的需求评估每一次发布,例如在你的黄金评估集上的质量表现以及你的延迟目标。

The Generative AI Lens 建议用你的基准真值数据(ground truth)测试候选模型,并具备快速更新模型的手段。更进一步,把这一决策当作一套经过测试的配置来对待:

把模型能力用在需要判断的地方,其他地方一概不用。你可以通过四个步骤做到这一点:

在本文中,你了解了如何判断某个工作流步骤到底需不需要模型。规则应放在确定性代码中,固定选择应放在受限决策调用中。开放式工作则交给规模合适的 Amazon Bedrock 模型。目标是在质量下限允许的范围内,以最低的成本和延迟满足每一项要求。在问该用哪个模型之前,先问这一步是否需要模型。

更多信息,请参阅 AWS Artificial Intelligence Blog 上的这些文章:

想尝试受限决策调用,可打开 Amazon Bedrock 控制台,或从 GitHub 仓库部署 Strands Decider。

从一个高流量工作流开始,衡量其中有多大比例其实从来不需要模型。

Meghan 是客户体验与业务趋势团队的高级技术产品经理,他与 AWS 领导层合作开展战略性的深度调研,以发现智能体式 AI 驱动的应用开发与现代化领域不断演变的趋势。他作为解决方案架构师和全栈开发者的背景,让他拥有独特的实践视角,从而帮助塑造开发者体验。

原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Cut generative AI costs: Code for rules, models for judgment)

The biggest cost lever in a generative AI system is whether you call a model at all. We often see teams route every task through a large language model (LLM), including input validation, formatting, and routing. Each call is cheap in isolation, but at production scale these small costs become one of the largest line items in the architecture. Wherever a deterministic rule can do the job, skipping the model saves money, cuts latency, and removes a source of non-determinism. In this post, you learn how to decide whether each step in a workflow needs a model at all. You can then map each option to AWS: deterministic code, a constrained decision call, or a right-sized model on Amazon Bedrock.

Why unneeded model calls add up

The pattern usually starts in a prototype. A model call works for one step, so it becomes the default for the next. Those steps include validating input, reformatting a request body, and picking a route. At prototype volume, nobody notices the cost.

At production volume, unneeded model calls cost you in three ways:

  • Spend grows with request count, even for low-value steps.
  • Latency stacks, because every model call in a chain adds hundreds of milliseconds.
  • Output can vary, because the same input can return a different answer. That makes testing harder and audits slower.

Each new model launch then restarts the debate about which model to use. The earlier question, whether the step needs a model at all, goes unasked.

Cost guidance, including the AWS Well-Architected Generative AI Lens, often starts at model selection. This post starts one step earlier: Does the step need a model at all?

How to decide whether a step needs a model

Figure 1 shows the whole decision as a flowchart with three labeled lanes: Runs as code, Questions, and Runs on a model. Take each workflow step through three questions in order, and stop at the first option that meets the step’s requirements:

  1. Can a rule check success, such as a format, a schema, a keyword, or a range? If a rule settles only part of the step, a fast path runs the rule first.
  2. Can a model write the rule once, such as a regular expression or a JSON schema?
  3. Does the step pick one answer from a fixed list, such as a route, a class, or yes or no?

If the answer to all three is no, the step is open-ended and runs on a right-sized model.

Flowchart that sorts a workflow step into code, a constrained decision call, or a right-sized Amazon Bedrock model

Figure 1: Deciding whether a workflow step needs a model, and which one

Consider a customer support assistant. The following table shows how five of its steps go through the chart.

# Step Path through the chart Where it runs
1 Check an order number’s format A rule can check success. Amazon API Gateway REST API request validation, or an AWS Lambda function
2 Route a message by intent A rule settles part of it. A keyword match on the fast path resolves messages such as “refund”, and the residual goes to the third question. An AWS Step Functions Choice state that uses JSONata. The residual goes to a constrained decision call.
3 Extract a tracking number from a carrier email A model can write the rule once. It drafts a regular expression, and you review and commit it. Code, as a committed pattern in AWS Lambda
4 Pick one of four support queues for a ticket The answer comes from a fixed list. A constrained decision call. Unsure answers escalate to the right-sized model.
5 Draft a reply to the customer The task is open-ended. The right-sized model, the smallest Amazon Bedrock model that meets the requirements

You can use a model when a task is open-ended, ambiguous, or generative: summarizing a document, drafting a reply, extracting meaning from prose. Use deterministic code when the task has a checkable rule: an email format, a schema, an exact keyword, a numeric range. The following table shows what you trade away when you send a rule-shaped task to a model.

Property Deterministic code Model call
Cost per invocation Typically orders of magnitude lower Per-token, non-trivial at scale
Latency Single-digit milliseconds or less Hundreds of milliseconds
Correctness on a well-specified rule Exact for the rule as written High, but with a non-zero error rate
Auditability The rule sits in source control Harder to explain to an auditor
Behavior over time Stable until you change it Changes when you move to a new model version, or when a version is retired

A model can also hallucinate and return different answers to the same input. A temperature of 0 makes varied answers less likely, but it doesn’t guarantee identical output. Amazon Bedrock model evaluation and Amazon Bedrock Guardrails help you measure and contain these risks. A rule has neither problem, which is one reason regulated workloads favor rules. A useful test: If you can write the acceptance test as a rule, you can usually write the step as a rule. For more information, refer to Orchestration models: From rule-based to AI-native in AWS Prescriptive Guidance.

Decide, don’t generate

Most routing and classification steps sit between a rule and open-ended generation. A rule can’t decide every case, but these steps don’t need generated text either. They need one answer from a fixed list of options that you define in advance, like the support queues in the preceding example. Use a component that limits its output to the allowed answers and gives you a signal for when to escalate. Confident answers proceed, and uncertain ones escalate. The AWS Well-Architected Agentic AI Lens describes this tiered, confidence-based escalation. On AWS, you can build it four ways:

  • A small Amazon Bedrock model with structured outputs – The answer is limited to an enum of options, plus an ‘unsure’ option that triggers escalation. This is the model’s own judgment, not a calibrated score, so check it against labeled data before you use it to decide escalation.
  • Embedding similarity – Embed the input with an Amazon Bedrock embeddings model and pick the nearest labeled example. Use the similarity score as an escalation signal, and set its threshold on labeled data.
  • Strands Decider, an experimental open source decision model under the Apache License 2.0 – It picks from options or scores against a rubric, and it returns a calibrated confidence with each answer. Its README reports that answers at a confidence of 0.9 or more are right about 95 percent of the time on short classification tasks. Because the model is experimental, validate that result on your own data before you use it in production. You host it yourself, for example on an Amazon SageMaker AI endpoint, by following the deployment guides in its repository.
  • A trained classifier on Amazon SageMaker AI – It returns class probabilities with low latency and cost once you have labeled training data.

The following JSON schema limits a structured-output answer to four queues plus ‘unsure’. Amazon Bedrock requires every object in the schema to set additionalProperties to false.

{
  "type": "object",
  "properties": {
    "queue": {
      "type": "string",
      "enum": ["billing", "technical", "account", "other", "unsure"]
    }
  },
  "required": ["queue"],
  "additionalProperties": false
}

Whichever option you choose, check its escalation signal against your own labeled data, and set any threshold there. Keep your authorization decisions, such as a refund limit, in a rule.

The deterministic fast-path pattern

Many cost patterns focus on routing between models. The fast path goes one level earlier: Run the deterministic check first, and send only what the rule can’t settle to a model. Intent routing is one example. Many messages carry an unambiguous signal, such as “refund” or “reset my password”, which a keyword match resolves in single-digit milliseconds or less. If rules resolve 95 percent of requests, only 1 in 20 requests reaches a model, before any savings from model selection. (Illustrative arithmetic, not a benchmark.)

On AWS, you compose this from existing building blocks:

  • AWS Step Functions Choice states branch before the workflow invokes any model.
  • AWS Lambda hosts rules that need more than a Choice state, such as validation and schema checks.
  • Amazon API Gateway (REST API) request validation rejects malformed input before any compute runs.
  • A constrained decision call or an Amazon Bedrock model handles only the residual.

The following Choice state is a keyword fast path. It uses JSONata to send refund and password-reset messages straight to code, and it sends everything else to the constrained decision call.

"KeywordFastPath": {
  "Type": "Choice",
  "QueryLanguage": "JSONata",
  "Choices": [
    {
      "Condition": "{% $contains($states.input.message, /refund/i) %}",
      "Next": "RefundFlow"
    },
    {
      "Condition": "{% $contains($states.input.message, /reset (my )?password/i) %}",
      "Next": "PasswordResetFlow"
    }
  ],
  "Default": "ConstrainedDecisionCall"
}

You can also let a model write the rule once instead of running it every time. Have it draft a regular expression or a JSON schema, review it, and commit it. At runtime the rule runs at rule speed and cost. Review a model-written regular expression, and test it against known good and bad inputs. Avoid nested quantifiers such as (a+)+, which can make matching very slow on some inputs (regular expression denial of service, or ReDoS).

When you do generate, Amazon Bedrock structured outputs constrain a response to a JSON schema that you define. The response then matches the schema, so your code can parse it without custom parsing logic (check content for correctness). Keep range checks, such as numeric bounds and length limits, in code. For supported models and schema features, refer to Get validated JSON results from models.

If your rules live in configuration, allow only known primitives, such as regular expressions, schema checks, keyword lists, and range checks. Never run arbitrary code from a configuration file.

When you do need a model, right-size it

Choose the cheapest and fastest model that meets your quality floor. That is often not the newest or largest model. The Generative AI Lens covers this in its best practice on right-sizing model selection. Effective cost optimization strategies for Amazon Bedrock covers it in depth. Define the requirements before you compare models:

  • Quality floor: The minimum acceptable output quality, measured on a golden evaluation set (labeled examples of the task with known-correct answers).
  • Cost ceiling: The most you pay per request or per successful outcome.
  • Latency target: The p95 or p99 response time the experience can tolerate.
  • Compliance constraints: Requirements such as data residency that can rule out candidates.

Amazon Bedrock offers many model families and sizes behind one API, so you can right-size each task without changing your integration. Amazon Bedrock Intelligent Prompt Routing routes each request to a suitable model within a supported model family. Amazon Bedrock Model Distillation transfers knowledge from a larger model to a smaller, cheaper one for your use case. Model providers evaluate a new frontier model on broad benchmarks, but your workload is one task with one quality floor. A larger model can add cost and latency without improving that task. Before you switch, evaluate each launch against your requirements, such as quality on your golden evaluation set and your latency target.

Treat model choice as tested configuration

The Generative AI Lens recommends testing candidate models against your ground truth data and having a way to update the model quickly. Go further and treat the decision as a tested configuration:

  • Keep a golden evaluation set per task, with labeled ground truth, and run it in continuous integration.
  • Gate on your requirements. Treat a cheaper model that drops below the quality floor as a regression.
  • Put the model behind a configuration flag so that you can reverse a swap.
  • Verify with the right oracle. Check rules against ground truth. Reserve model-as-judge evaluation for open-ended outputs.
  • Watch for drift. Sample some rule-resolved requests through a model in shadow mode, and reclassify a step when they disagree.

Putting it into practice

Spend model capacity where judgment is needed, and nowhere else. You can get there in four steps:

  1. Instrument – Tag every call as deterministic code, a constrained decision call, or a model call. Track cost per successful outcome.
  2. Move the rules to code – Run rule-checkable steps on AWS Lambda or AWS Step Functions. This is usually the largest and safest win.
  3. Add constrained decision calls – Route fixed-choice steps through a call that escalates when it is unsure.
  4. Test before you switch models – Run every new model through your golden evaluation set before it becomes the default.

Conclusion

In this post, you learned how to decide whether a workflow step needs a model at all. Rules belong in deterministic code, and fixed choices belong in a constrained decision call. Open-ended work goes to a right-sized Amazon Bedrock model. Aim to meet each requirement at the lowest cost and latency that your quality floor allows. Ask whether a step needs a model before you ask which one.

For more information, refer to these posts on the AWS Artificial Intelligence Blog:

To try a constrained decision call, open the Amazon Bedrock console, or deploy Strands Decider from its GitHub repository.

Start with one high-volume workflow and measure how much of it never needed a model.

 


About the authors

Meghan Kothari

Meghan Kothari

Meghan is a Senior Technical Product Manager with the Customer Experience and Business Trends team, where he partners with AWS leadership on strategic deep dives to discover evolving trends in agentic AI-driven application development and modernization. His background as a solutions architect and full-stack developer gives him a unique hands-on perspective to help shape the developer experience.

Krishna Aswani

Krishna Aswani

Krishna is a Senior Applied Scientist at Amazon with more than a decade of experience building machine learning systems that run in production. His work covers computer vision, natural language processing, generative modeling, and large language models. At Amazon, he builds agentic architectures and leads research on structured reasoning frameworks that make LLMs more reliable and interpretable. In a prior life, his research focused on neurocomputing and biophysics. Working with the University of Oxford and NIMS Japan, he built simulators and measurement systems to understand how information propagates through brain microtubules, including the potential role of quantum effects in these biological structures.

Pranav Patankar

Pranav Patankar

Pranav is a Science and Engineering Manager with the Customer Experience and Business Trends team, where he partners with AWS leadership on improving the developer experience on AWS. His research focuses on enhancing the reasoning abilities of LLMs and AI agents and on making them more trustworthy. His background spanning data engineering, data science, and applied research gives him a unique hands-on perspective to help shape the developer experience.

Vineet Kamdar

Vineet Kamdar

Vineet leads a multi-functional team evaluating cloud, agentic developer experiences and overall customer experiences. He leads strategic evaluations and deep dives to influence rapidly evolving generative AI, agentic developer and customer experiences for areas such as agentic coding assistants, agentic migrations, model training, inference. His 18+ years of industry experience, extensive background as a builder, architect, and product leader for enterprise large-scale applications uniquely positions him to evaluate, measure, influence and improve developer, agentic, and overall cloud customer experience.

正文保留英文原文,标题与摘要由机器翻译。

本文由采集器自动同步

原文发布于 AWS Machine Learning(Cut generative AI costs: Code for rules, models for judgment),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗