影响成本的四个 Amazon Bedrock 推理选项

即使优化了提示词和应用逻辑,你的 Amazon Bedrock 推理账单仍可能持续攀升。请求级选择仍会改变服务处理的 Token 数量以及适用的费率。缓存持续时间、服务层级、推理强度和 maxTokens 都可能在不改变模型或提示词的情况下影响成本。
每种选择都取决于模型和工作负载。本文展示我们如何测试每种选择,以及如何从响应元数据、日志和账单数据中验证其效果。
要自行尝试这些设置,请从 Amazon Bedrock Samples 笔记本 Mixed TTL Caching 和 Tenant Isolation with Prompt Caching 开始。关于 maxTokens 和推理强度,请参阅 GitHub 仓库 sample-code-for-genai-cost-management-workshop 的 01-low-effort 文件夹。每个示例都设置了自己的模型 ID,因此请更改它以及推理参数,以匹配你调用的模型。关于各 AWS 区域的模型可用性,请参阅 Models at a glance。对于模型选择、批量推理、路由和蒸馏等更广泛的决策,请参阅 Effective cost optimization strategies for Amazon Bedrock。
每个 Amazon Bedrock 响应都带有用量元数据。输入和输出 Token 按处理该请求的模型和层级的费率计费。缓存读取和写入使用单独的 Token 计数器,推理 Token 计为输出 Token。
当下一次请求前缓存已过期时,重复的系统指令、工具定义或参考上下文可能会被再次处理。Prompt caching for faster model inference 指南列出了受支持的 Anthropic 模型的 5 分钟和 1 小时生存时间(TTL)值。它还列出了受支持的 GPT-5.6 模型的 30 分钟最低 TTL。支持和 API 字段取决于模型,因此在选择持续时间之前,请查看当前支持矩阵。
对于受支持的模型,Amazon Bedrock 还提供隐式提示缓存(implicit prompt caching),它会在你的请求中不含缓存控制的情况下,尝试复用符合条件的提示前缀。由于隐式缓存是尽力而为,并不保证缓存命中,因此本节中的时长选项适用于显式缓存。
以下 Converse API 签名会在支持该功能的模型上请求一个 1 小时的条目:
{ "cachePoint": { "type": "default", "ttl": "1h" } } 对于受支持的 Anthropic 模型,InvokeModel 会将相同的 ttl 值放入 cache_control。
更长的时长并不会自动更便宜。Amazon Bedrock 定价页面分别列出了 5 分钟写入、1 小时写入和缓存读取的费率。对于 Amazon Bedrock 上的 Anthropic Claude Sonnet 5 和 Anthropic Claude Opus 5,5 分钟写入的费用是输入费率的 1.25 倍,1 小时写入是 2 倍,缓存读取是 0.1 倍。
这些费率划定了盈亏平衡点,前提是每个请求都复用完全相同的前缀,且每次缓存读取都会刷新 TTL。5 分钟缓存只要在 5 分钟内首次复用即可回本。当同一前缀的请求间隔超过 5 分钟才到达时,每个请求都会以 1.25 倍的价格重新写入。此时,1 小时缓存在一小时内从第二次请求起就更便宜。
缓存费率和检查点最小值因模型而异。请查阅 Amazon Bedrock 定价页面,以及你所调用模型的提示缓存指南。针对你的模型重新运行这项对比,并在上线后通过缓存读取和写入 Token 加以确认。
将稳定内容放在动态内容之前,满足模型的检查点最小值要求,并使用按需推理,因为提示缓存不适用于批量推理。
修改已缓存的前缀——无论是系统指令还是工具定义——都会导致缓存未命中和一次新的写入。检查点按 tools、system、messages 的顺序依次评估,因此更靠前部分的改动也会使后面的部分失效。对经常变化的提示词做版本管理,让每个版本拥有自己的缓存条目。我们在 Claude Sonnet 5 上以 1 小时检查点运行了六次调用,期间各改动了一次系统提示词和一次工具定义。每次改动都产生了零次缓存读取和一次新的写入。
AWS 将缓存条目的作用域描述为账户和 Region 级别,而非租户级别。在多租户应用中,要在可缓存前缀前加上特定于租户的 SHA-256 哈希。《Optimizing cost and latency with Amazon Bedrock prompt caching for Anthropic models》一文描述了这一模式。在构建提示词之前先执行租户鉴权。该哈希在一个账户内对条目进行分区。如果受监管的工作负载需要严格的数据边界,就把每个租户放在单独的 AWS 账户中。
对于频繁复用的场景,保留 5 分钟的默认值。当同一前缀每小时至少重复出现两次、且间隔超过 5 分钟时,再改为受支持的更长时长。要想胜过完全不做缓存,1 小时缓存每小时需要三次请求。
后台任务、评估框架、夜间摘要和回填并不总是需要交互式请求所用的延迟特征。对于支持 Flex 的模型,可通过以下请求签名选择它:
{ "serviceTier": { "type": "flex" } } 要请求 Standard 层级,请使用 default,而不是 standard。你也可以省略 serviceTier:
{ "serviceTier": { "type": "default" } } 层级支持因模型而异,因此要按确切的模型版本来记录支持情况。Amazon Nova 2 Lite 和 OpenAI gpt-oss-120b 的模型卡片将 Flex 列为支持。而 Amazon Bedrock 上 Anthropic Claude Sonnet 5 和 Anthropic Claude Opus 5 的模型卡片则将其列为不支持。向不支持的模型发送 Flex 请求会以 ValidationException 失败,而不是静默地按 Standard 运行。我们向 Anthropic Claude Sonnet 5 发出的请求就返回了该错误。
如果你的策略允许以 Standard 作为回退,就捕获“不支持该层级”的错误,并用 default 重试一次。不要对每个 ValidationException 都重试,因为格式错误的请求也会引发该异常。要为应用层重试设置上限。AWS SDK 的有界重试策略会处理限流和瞬时错误。
为 Flex 请求设置你自己的截止时间。受支持的响应会报告最终解析出的层级,你也可以通过 Amazon CloudWatch 的 ResolvedServiceTier 进行核实。关于优化性能与成本,请参阅 Service tiers;关于你所调用具体模型的费率,请参阅 Amazon Bedrock Pricing 页面。
我们在 Amazon Nova 2 Lite 上测试了 Flex:200 个 default 请求和 200 个 flex 请求,以交错成对的方式发送。两个层级处理的输入和输出 Token 数量相同。全部 400 个请求都解析到了所请求的层级,延迟中位数的差异小到无法与噪声区分。截至撰稿时,us-west-2 区域公布的 Nova 2 Lite 的 Flex 费率是 Standard 费率的一半,我们的账单也反映了这一点。请查看 Amazon Bedrock Pricing 页面了解当前费率。对延迟敏感的流量请继续使用 default。
推理 Token 按输出 Token 计费。Claude Sonnet 5 和 Claude Opus 5 默认在高 effort 下使用自适应思考,即使请求中省略了 thinking 字段也是如此。一个未做任何改动就从 Claude Sonnet 4.6 迁移过来的应用会开始为思考 Token 计费,而且 maxTokens 现在同时限制思考内容与回复文本。Claude Haiku 5.5 默认使用 medium。这三个模型都支持 low、medium、high、xhigh 和 max。对于具有以下请求签名的任务,请调低 effort:
{ "thinking": { "type": "adaptive" }, "output_config": { "effort": "low" } } 请把 effort 放在 output_config 中,而不是放在 thinking 内部。每次更换模型时都要检查自适应思考设置。
我们在 Claude Opus 5 上测试了一个工具调用智能体。它的三个任务遵循《Build long-horizon agents with Grok 4.7 on Amazon Bedrock》中的设计,每个任务都有一个正确答案。我们在每一个 effort 级别下把每个任务各运行了八次。下表显示的是输出 Token 的中位数:
较低 effort 下的准确率取决于模型和任务。请在与你的任务形态相同的任务上进行测量。
在 Amazon Bedrock Samples 的 Invoke Bedrock model for code generation 工作负载上,输出 Token 中位数在 low 下为 676,在 high 下为 743,这一差异处于运行间波动范围之内。我们评分的程序在每个 effort 级别下都通过了。
请使用按请求统计的输出 Token 数量、CloudWatch 的 AWS/Bedrock OutputTokenCount、调用日志以及任务通过率来评估这一选择。对于常规、边界清晰的任务,从低于 high 的级别开始。如果你的质量测试表明任务需要 high,就保持 high。省下 Token 却让任务失败,算不上节省。
maxTokens 是应用程序的输出上限。它限制了错误提示词、回归或循环在请求停止之前可能产生的输出。配额规则记录在《How tokens are counted in Amazon Bedrock》中。请求开始时,Amazon Bedrock 会扣除你的输入 Token 加上 maxTokens,因此设置过大的值会降低并发能力。
输出上限并不是一种安全控制。在生产环境的生成式 AI 工作负载中,应将 Amazon Bedrock Guardrails 用作标准的负责任 AI 控制措施。Guardrails 可以过滤内容、检测提示攻击,并在支持的输入和响应上检查 grounding。参见《Detect and filter harmful content by using Amazon Bedrock Guardrails》。
在观察到具有代表性的输出 Token 分布后,按任务类型设置 maxTokens:
{ "inferenceConfig": { "maxTokens": 155 } } 请求参数参见《Inference using Converse API》。
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Four Amazon Bedrock inference choices that affect cost)
Your Amazon Bedrock inference bill can keep climbing after you tune prompts and application logic. Request-level choices can still change how many tokens the service processes and the rate applied to them. Cache duration, service tier, reasoning effort, and maxTokens can affect cost without changing your model or prompt.
Each choice depends on the model and workload. This post shows how we tested each choice and how to verify its effect from response metadata, logs, and billing data.
To try these settings yourself, start with the Amazon Bedrock Samples notebooks Mixed TTL Caching and Tenant Isolation with Prompt Caching. For maxTokens and reasoning effort, see the 01-low-effort folder of the sample-code-for-genai-cost-management-workshop GitHub repo. Each sample sets its own model ID, so change it, and the inference parameters, to match the model you call. For model availability by AWS Region, refer to Models at a glance. For broader decisions such as model selection, batch inference, routing, and distillation, see Effective cost optimization strategies for Amazon Bedrock.
Where inference cost comes from
Every Amazon Bedrock response carries usage metadata. Input and output tokens bill at the rate for the model and tier that served the request. Cache reads and writes use separate token counters, and reasoning tokens count as output tokens.
Prompt cache duration
Repeated system instructions, tool definitions, or reference context can be processed again when the cache expires before the next request. The Prompt caching for faster model inference guide lists 5-minute and 1-hour time-to-live (TTL) values for supported Anthropic models. It also lists a 30-minute minimum TTL for supported GPT-5.6 models. Support and API fields depend on the model, so check the current support matrix before selecting a duration.
For supported models, Amazon Bedrock also offers implicit prompt caching, which attempts to reuse eligible prompt prefixes without cache controls in your request. Because implicit caching is best effort and doesn’t guarantee a cache hit, the duration choices in this section apply to explicit caching.
This Converse API signature requests a 1-hour entry on models that support it:
{ "cachePoint": { "type": "default", "ttl": "1h" } }
For supported Anthropic models, InvokeModel places the same ttl value in cache_control.
A longer duration is not automatically cheaper. The Amazon Bedrock Pricing page lists separate rates for 5-minute writes, 1-hour writes, and cache reads. For Anthropic Claude Sonnet 5 and Anthropic Claude Opus 5 on Amazon Bedrock, a 5-minute write costs 1.25 times the input rate. A 1-hour write costs 2 times, and a cache read costs 0.1 times.
These rates set the break-even, assuming each request reuses the exact prefix and each cache read refreshes the TTL. A 5-minute cache pays back on the first reuse within 5 minutes. When requests for the same prefix arrive more than 5 minutes apart, it rewrites at 1.25 times on every request. The 1-hour cache is then cheaper from the second request within the hour.
Cache rates and checkpoint minimums differ by model. Check the Amazon Bedrock Pricing page and the prompt caching guide for the model you call. Rerun the comparison for your model, then confirm it with cache read and write tokens after rollout.
Keep stable content before dynamic content, meet the model’s checkpoint minimum, and use on-demand inference because prompt caching doesn’t work with batch inference.
Changing a cached prefix, including a system instruction or tool definition, causes a miss and a new write. Checkpoints are evaluated in the order tools, system, then messages, so a change in an earlier section also invalidates the later ones. Version prompts that change often so each version gets its own cache entry. We ran six calls on Claude Sonnet 5 with a 1-hour checkpoint, changing the system prompt once and the tool definition once. Each change produced zero cache reads and a new write.
AWS describes cache entries as scoped to your account and Region, not to your tenants. In a multi-tenant application, prepend a tenant-specific SHA-256 hash to the cacheable prefix. The Optimizing cost and latency with Amazon Bedrock prompt caching for Anthropic models post describes this pattern. Enforce tenant authorization before you build the prompt. The hash partitions entries inside one account. If a regulated workload needs a hard data boundary, keep each tenant in a separate AWS account.
Keep the 5-minute default for frequent reuse. Move from it to a longer supported duration when the same prefix recurs at least twice an hour with gaps longer than 5 minutes. To beat no caching at all, the 1-hour cache needs three requests an hour.
Service tiers for latency-tolerant workloads
Background jobs, evaluation harnesses, nightly summarization, and backfills don’t always need the latency profile used by interactive requests. For models that support Flex, select it with this request signature:
{ "serviceTier": { "type": "flex" } }
Use default, not standard, to request the Standard tier. You can also omit serviceTier:
{ "serviceTier": { "type": "default" } }
Tier support is model-specific, so record support by exact model version. The Amazon Nova 2 Lite and OpenAI gpt-oss-120b model cards list Flex as supported. The cards for Anthropic Claude Sonnet 5 and Anthropic Claude Opus 5 on Amazon Bedrock list it as not supported. A Flex request to an unsupported model fails with a ValidationException instead of silently running on Standard. Our request to Anthropic Claude Sonnet 5 returned that error.
If your policy allows Standard as a fallback, catch the unsupported-tier error and retry once with default. Don’t retry every ValidationException, because malformed requests raise it too. Cap application-level retries. The bounded retry strategy of the AWS SDK handles throttling and transient errors.
Set your own deadline for Flex requests. A supported response reports the resolved tier, which you can also verify with Amazon CloudWatch ResolvedServiceTier. See Service tiers for optimizing performance and cost and the Amazon Bedrock Pricing page for the exact model you call.
We tested Flex on Amazon Nova 2 Lite with 200 default requests and 200 flex requests, sent in interleaved pairs. Both tiers processed the same number of input and output tokens. All 400 requests resolved to the requested tier, and the median latency difference was too small to separate from noise. At the time of writing, the published Flex rate for Nova 2 Lite in us-west-2 is half the Standard rate, and our bill reflected that. Check the Amazon Bedrock Pricing page for current rates. Keep latency-sensitive traffic on default.
Adaptive thinking effort
Reasoning tokens bill as output tokens. Claude Sonnet 5 and Claude Opus 5 use adaptive thinking at high effort by default, even when a request omits the thinking field. An application moved from Claude Sonnet 4.6 without changes starts billing thinking tokens, and maxTokens now caps thinking plus response text. Claude Haiku 5.5 defaults to medium. All three models support low, medium, high, xhigh, and max. Lower the effort for a task with this request signature:
{
"thinking": { "type": "adaptive" },
"output_config": { "effort": "low" }
}
Keep effort in output_config, not inside thinking. Check Adaptive thinking whenever you change models.
We tested a tool-calling agent on Claude Opus 5. Its three tasks follow the design in Build long-horizon agents with Grok 4.7 on Amazon Bedrock, and each has one correct answer. We ran each task eight times at every effort level. The table shows median output tokens:
| Effort | Correct runs | Median output tokens | Change from low (95% interval) |
| low | 24 of 24 | 936 | baseline |
| medium | 24 of 24 | 1,484 | +59% (50% to 78%) |
| high (default) | 24 of 24 | 2,521 | +169% (112% to 190%) |
| xhigh | 24 of 24 | 2,964 | +217% (170% to 260%) |
| max | 24 of 24 | 3,762 | +302% (180% to 402%) |
Accuracy at lower effort depends on the model and the task. Measure on tasks shaped like yours.
On the Amazon Bedrock Samples Invoke Bedrock model for code generation workload, median output tokens were 676 at low and 743 at high, a difference within run-to-run variation. The programs we graded passed at each effort level.
Use per-request output-token counts, CloudWatch AWS/Bedrock OutputTokenCount, invocation logs, and a task pass rate to evaluate the choice. Start below high for routine, well-bounded tasks. Keep high where your quality tests show that the task needs it. A token reduction that breaks the task is not a saving.
Output token limits
maxTokens is an application output ceiling. It limits the output that a bad prompt, regression, or loop can produce before the request stops. The quota rule is documented in How tokens are counted in Amazon Bedrock. Amazon Bedrock deducts your input tokens plus maxTokens when a request starts, so an oversized value reduces concurrency.
An output ceiling isn’t a safety control. Use Amazon Bedrock Guardrails as a standard responsible AI control in production generative AI workloads. Guardrails can filter content, detect prompt attacks, and check grounding on supported inputs and responses. See Detect and filter harmful content by using Amazon Bedrock Guardrails.
Set maxTokens per task type after observing a representative output-token distribution:
{ "inferenceConfig": { "maxTokens": 155 } }
See Inference using Converse API for the request parameters.
In our test on Claude Sonnet 5, five normal outputs peaked at 103 tokens, so we set an illustrative fence of 155 tokens with headroom. We then sent the same deliberately verbose prompt with adaptive thinking disabled, once under a 2,048-token ceiling and once under the 155-token fence. The first request generated 2,048 output tokens and the second generated 155, so the fence avoided 1,893 output tokens. Both stopped at max_tokens, so both responses were truncated. The fence limits spend but does not make an unsuitable prompt useful, and in a multi-turn or agentic chain a truncated turn can break the steps after it.
For production calibration, compare the 95th and 99th percentiles (P95 and P99) for each task type and add headroom to the selected percentile. With adaptive thinking on, maxTokens also caps thinking tokens, so calibrate with thinking enabled. P95 limits exposure more aggressively but accepts more truncation risk. P99 preserves more valid long-tail responses at a higher ceiling. Monitor max_tokens stops and investigate the request before raising the task’s limit.
Putting it together
Use the same test loop for each setting: confirm model support, change one request choice, measure tokens or cost, and track quality or completion. If you route across models, keep a capability record for each model version, Region, and API: supported cache duration, service tiers, reasoning levels, and output ceiling. Validate each request against that record before you send it, then check the response for the resolved tier and cache counters.
| Setting | Default or permissive state | Result | Risk to accept |
| Cache duration | Model-specific TTL | From published Sonnet 5 and Opus 5 rates, a 1-hour cache wins from the second request per hour when gaps exceed 5 minutes | A 1-hour write costs 2 times input, so low reuse loses money. Partition tenant prefixes |
| Service tier | default |
At the time of writing, Nova 2 Lite Flex was published at half the Standard rate, and our bill matched | Longer processing. Set your own deadline and a fallback |
| Reasoning effort | high on the tested adaptive model |
Low to high: +169% median output tokens on the agent test, no reliable difference on code generation | Quality can drop near the task’s capability boundary. Track pass rate with tokens |
maxTokens |
Permissive 2,048-token cap | An illustrative 155-token fence avoided 1,893 output tokens in one test | Truncation can cascade through multi-turn or agentic chains. Check stopReason before raising the limit |
Signals to watch
| Setting | Symptom | Verify with | First response |
| Cache duration | Repeated context does not produce the expected latency or cost benefit | Application latency and cost telemetry | Check prefix stability, reuse interval, and model support before changing TTL |
| Service tier | The requested tier is unsupported or the serving tier differs from expectation | Response serviceTier and CloudWatch ResolvedServiceTier |
Use a supported tier or return latency-sensitive traffic to default |
| Reasoning effort | Output tokens rise while quality holds | Per-request response telemetry, CloudWatch OutputTokenCount, invocation logs, and task pass rate |
Lower effort for that task class and rerun its quality test |
maxTokens |
stopReason is max_tokens |
Response stop reason and task-scoped output-token distribution | Inspect the prompt and task fence before raising the limit |
Derive P95 and P99 for each task type from task-tagged application telemetry or invocation logs.
Conclusion
All four choices live in the inference request. Recheck support when you change models, then change one setting at a time and measure the result in your application.
Use these resources to evaluate the choices for your workload:
- Amazon Bedrock console: console.aws.amazon.com/bedrock.
- Prompt caching: Prompt caching for faster model inference.
- Service tiers: Service tiers for optimizing performance and cost.
- Adaptive thinking: Adaptive thinking.
- Responsible AI controls: Detect and filter harmful content by using Amazon Bedrock Guardrails.
- Related reading: Effective cost optimization strategies for Amazon Bedrock.
- Related samples and posts: Invoke Bedrock model for code generation and Build long-horizon agents with Grok 4.7 on Amazon Bedrock.
About the authors
Sai Pavan Kasani
Sai is a Technical Account Manager at AWS. As a technical advisor and customer advocate, he helps customers move generative AI proofs of concept into production, with a focus on agentic AI workloads, inference optimization, and cost efficiency at scale. Outside of work, Sai enjoys exploring new coffee shops and spending time with family.
Aswath Ram Srinivasan
Aswath is a Senior Worldwide Specialist Solutions Architect for Generative AI at Amazon Web Services, focused on helping customers build generative AI applications with Amazon Bedrock. He works with enterprise teams to design inference architectures that balance performance, cost, and operational simplicity.
正文保留英文原文,标题与摘要由机器翻译。
原文发布于 AWS Machine Learning(Four Amazon Bedrock inference choices that affect cost),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。