Accenture 分阶段方法优化 Bedrock AI 成本

随着企业在 Amazon Bedrock 上扩展 AI 应用,即便单 Token 价格下降,Token 支出往往仍会增长。如果没有设置成本分摊标签,你的云账单只会把 AI 支出显示为单一账单项,而无法按团队、工作负载、智能体或模型拆分。更便宜的 Token 会推动采用范围扩大,而智能体式工作负载会使调用次数成倍增加。随着成功的 AI 用例规模扩大,消耗量的增长可能快于单价的下降。经济层面的问题在于,所产生的业务价值是否比产出这些智能所需的成本增长得更快。Linux Foundation 已发起 Tokenomics Foundation,以制定 AI 成本经济学的开放标准,这表明管理 AI 的经济性如今已成为一项行业学科。
在这篇文章中,你将了解优化 Token 支出的六种手段、Accenture 如何在生产环境中应用它们,以及如何使用 Amazon Bedrock 应用它们。
Accenture 在电信、保险、采矿以及一个大型企业部署这四类客户合作中应用了这些手段。如本文所述,这些手段在 Amazon Bedrock 上原生可用。在衡量了节省效果的情况下,Token 支出下降了约 40–85%。结果因每个客户的工作负载而异,可能会有所不同。为保护客户机密,Accenture 未披露客户名称。
在你优化任何东西之前,你需要先看清消耗发生在哪里、由谁负责、产生了什么业务价值。团队、工作负载和智能体层级的成本分配标签,是把消耗与可问责的负责人关联起来、并最终与业务成果关联起来的前提。你可以通过 Amazon Bedrock 应用推理配置文件来应用它们,每个应用或工作负载一个配置文件,并按团队、成本中心和环境打标签。在 AWS Billing and Cost Management 控制台中将其激活为成本分配标签后,你就可以用这些标签追踪哪些团队、应用和智能体在推动支出。它们还让你能够衡量你引入的每一项变更所带来的影响。
三项 AWS 服务提供底层遥测数据。通过 AWS Data Exports 交付的 AWS Cost and Usage Reports(CUR 2.0)负责财务归因,在你的成本分配标签生效后,按工作负载和团队细分计费的推理成本。Amazon CloudWatch 捕获调用次数和延迟等运行信号。Amazon Bedrock 模型调用日志记录请求级细节,包括每次调用所使用的模型以及输入和输出 Token 数量。
仅靠遥测数据还不够。工程师可以在仪表盘上看到每次调用的成本,但产品负责人和项目负责人需要一个用业务语言表达的指标。Accenture 的每笔业务交易成本(Cost per Business Transaction,CpBT)框架弥合了这一差距。CpBT 将促成一次成功业务产出的所有模型调用、重试、工具调用和智能体交接,汇总为一个完全成本口径的单一数字。它能揭示产出结果的真实成本,包括原始 Token 定价所掩盖的失败尝试。将 CpBT 与每个工作流所交付的业务价值对照追踪,就能区分出一张增长中的账单究竟反映的是规模化带来的成功,还是反映的浪费。
这六个杠杆的目标并不是不惜一切代价最小化 Token 用量,而是在恰当的成本、质量和风险下,使用合适数量和级别的智能来交付每一项业务成果。这些杠杆按照变革的复杂度和所需投入分为三个阶段。AWS 和 Accenture 建议按顺序推进这些阶段,你也可以将每个阶段单独应用到它能产生最大影响的工作负载上。
借助 Amazon Bedrock 的提示缓存(prompt caching),模型会复用请求之间重复出现的提示内容部分,例如系统指令、工具定义和参考文档,这样你就不必再为处理相同内容支付全额费用。Amazon Bedrock 会存储此前已计算过的提示前缀的键值状态,并在后续请求中跳过重新计算。对于受支持的模型,该功能可将缓存输入 Token 的成本降低最多 90%(参见 Amazon Bedrock Prompt Caching)。
Amazon Bedrock 批处理推理以异步方式将非紧急工作负载作为一个批次处理,对部分模型而言,其成本比按需定价低 50%(参见 Amazon Bedrock 定价)。你将推理请求作为批处理作业提交,处理完成后结果会写入 Amazon Simple Storage Service(Amazon S3)。批处理推理适用于夜间报告、文档处理流水线以及无需立即响应的评估运行。
这两种手段都是在现有配置基础上增加设置。提示缓存需要在提示中加入缓存检查点标记,批处理推理则要求将符合条件的请求作为批处理作业提交,而不是调用按需 API。缓存适用于交互式的按需流量,批处理推理适用于异步工作。两者都不需要新的基础设施或架构变更。
Amazon Bedrock 智能提示路由会将请求分发到同一系列内的不同模型,让每项任务匹配其业务价值、复杂度和风险所要求的智能水平,而不是把每个请求都默认交给最强大的模型。智能提示路由最多可将成本降低 30%,且不牺牲准确性(参见 Amazon Bedrock 智能提示路由)。路由是否收费请查看 Amazon Bedrock 定价。在接下来的项目中,Accenture 的模型分层与分诊路由采用了同样的路由手段。
检索增强生成(RAG)优化会调整 RAG 流程中的检索步骤,使更少不必要的上下文 Token 进入模型。分层分块、基于元数据的过滤和重排序等技术可将检索到的上下文收窄至最相关的分块,因此你主要为有助于提升答案质量的内容付费。
两种互补的 Accenture 模式与这些 AWS 原生手段配合使用。上下文范围限定(Context scoping)将每个智能体接收的数据限制在工作流中与其步骤相关的文档片段,而不是在每次交接时传递完整的案例上下文。上下文压缩(Context compression)则在文档到达模型之前对其进行摘要或过滤。两者都能减少输入 Token 量,在多智能体链中尤为明显。
语义缓存(Semantic caching)会存储响应,并在新请求与已作答的请求语义相似时复用这些响应,即使措辞不同。这避免了冗余的模型调用,同时降低成本与延迟。在 AWS 上,你可以在模型前置一个向量存储(例如 Amazon ElastiCache)来实现它。在一项针对 63,796 条真实聊天机器人查询的 AWS 基准测试中,0.90 的相似度阈值带来了 74.5% 的缓存命中率、92.3% 的准确率以及 72.5% 的成本节省(参见 ElastiCache 语义缓存基准测试)。在应用路由和语义缓存时,请在每条路由上启用 Amazon Bedrock Guardrails,并设置语义缓存过期时间,以免用户收到过时的答案。
模型蒸馏(Model distillation)将知识从大型教师模型迁移到更小、更快的学生模型。Amazon Bedrock 中的蒸馏模型速度最高提升 500%,成本最高降低 75%,而在检索增强生成(RAG)等用例中准确率损失不到 2%。蒸馏最适合稳定、高吞吐量的工作负载,这类场景的交易量足以支撑训练投入。
Accenture 在四个客户项目中应用了这些手段。每个项目针对不同的成本模式。其中两个项目中,Accenture 同时应用了多项改动,因此这些结果反映的是综合效果,而非任何单一手段的效果。
第一个项目是一家大型企业的部署,其中每个请求都会发往前沿模型,没有缓存、模型分层或批处理。Accenture 同时实施了三项改动:模型分层,把每项任务匹配到能够胜任的最小模型;针对重复上下文的提示缓存;以及对夜间和非紧急任务的批处理。三者合计将每月 Token 成本降低了 85%,而且这一节省并未按改动项拆分。约有 15% 的任务仍需要顶级模型来完成高难度的开放式推理,其余任务则运行在更轻量的模型上。你的比例将取决于自身工作负载的构成。
第二个项目是一家电信运营商,年度 Token 支出为 1200 万美元。这笔支出大部分来自智能体无限制地扇出请求,以及无论复杂程度如何、每个请求都发往前沿模型。Accenture 同时实施了分诊路由、语义缓存和上下文压缩。输入 Token 减少了 70%,年度 Token 支出从 1200 万美元降至 380 万美元,降幅为 68%。Accenture 依据该运营商改动前的性能基准测试验证了输出质量。
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Optimize AI economics on Amazon Bedrock with Accenture’s sequenced approach)
As enterprises scale AI on Amazon Bedrock, token spend often grows even as per-token prices fall. Without cost allocation tags in place, your cloud invoice shows AI spend as a single line item rather than by team, workload, agent, or model. Cheaper tokens expand adoption, and agentic workloads multiply invocations. As successful AI use cases scale, consumption can rise faster than unit prices drop. The economic question is whether the business value generated grows faster than the cost of the intelligence required to produce it. The Linux Foundation has launched the Tokenomics Foundation to develop open standards for AI cost economics, a sign that managing the economics of AI is now an industry discipline.
In this post, you will learn about six levers for optimizing token spend, how Accenture applied them in production, and how to apply them using Amazon Bedrock.
Accenture applied these levers in four client engagements across telecom, insurance, mining, and a large enterprise deployment. The same levers are available natively on Amazon Bedrock as described in this post. Where savings were measured, token spend fell by about 40–85 percent. Results are specific to each client’s workload and may vary. Client names are withheld by Accenture to protect client confidentiality.
Attribution first: Measuring cost at the business level
Before you optimize anything, you need to see where consumption is occurring, who owns it, and what business value it produces. Cost allocation tags at the team, workload, and agent level are the prerequisite for connecting consumption to accountable owners and, ultimately, to business outcomes. You apply them through Amazon Bedrock application inference profiles, one profile per application or workload, tagged by team, cost center, and environment. After you activate them as cost allocation tags in the AWS Billing and Cost Management console, you can use these tags to trace which teams, applications, and agents are driving spend. They also make it possible to measure the impact of each change you introduce.
Three AWS services provide the underlying telemetry. AWS Cost and Usage Reports (CUR 2.0), delivered through AWS Data Exports, handle financial attribution, breaking down billed inference cost by workload and team after your cost allocation tags are active. Amazon CloudWatch captures operational signals such as invocation counts and latency. Amazon Bedrock model invocation logging records request-level detail, including the model used and the input and output token counts for each call.
Telemetry alone isn’t enough. Engineers can read cost per invocation on a dashboard, but product owners and program leaders need a metric expressed in business terms. Accenture’s Cost per Business Transaction (CpBT) framework bridges this gap. CpBT aggregates all the model invocations, retries, tool calls, and agent handoffs that contribute to one successful business output into a single fully loaded cost figure. It surfaces the true cost of producing a result, including the failed attempts that raw token pricing hides. Tracked against the business value each workflow delivers, CpBT distinguishes a growing bill that reflects scaling success from one that reflects waste.
The six optimization levers
The goal of these six levers isn’t to minimize tokens at all costs. It’s to use the right amount and level of intelligence to deliver each business outcome at the appropriate cost, quality, and risk. The levers are organized into three phases based on the complexity of the change and the investment it requires. AWS and Accenture recommend working through the phases in order, and you can apply each phase independently to the workloads where it will have the most impact.
Phase 1: Configuration only, no infrastructure change
With Amazon Bedrock prompt caching, the model reuses the parts of a prompt that repeat across requests, such as system instructions, tool definitions, and reference documents, so you stop paying full price to process the same content again. Amazon Bedrock stores the key-value state of previously computed prompt prefixes and is designed to skip recomputation on later requests. This is designed to reduce cached input token costs by up to 90 percent for supported models (see Amazon Bedrock Prompt Caching).
Amazon Bedrock batch inference processes non-urgent workloads asynchronously as a group at 50 percent lower cost than on-demand pricing for select models (see Amazon Bedrock Pricing). You submit inference requests as a batch job, and results are written to Amazon Simple Storage Service (Amazon S3) when processing completes. Batch inference suits nightly reports, document processing pipelines, and evaluation runs that don’t need an immediate response.
Both levers add configuration to your existing setup. Prompt caching requires cache checkpoint markers in your prompts, and batch inference requires submitting eligible requests as a batch job instead of calling the on-demand API. Apply caching to interactive, on-demand traffic and batch inference to asynchronous work. Neither requires new infrastructure or architecture changes.
Phase 2: Architectural changes
Amazon Bedrock intelligent prompt routing directs requests across models within the same family, matching each task to the level of intelligence its business value, complexity, and risk require, rather than defaulting every request to the most powerful model. Intelligent prompt routing can reduce costs by up to 30 percent without compromising accuracy (see Amazon Bedrock intelligent prompt routing). Check Amazon Bedrock Pricing for any routing charges. In the following engagements, Accenture’s model tiering and triage routing apply this same routing lever.
Retrieval Augmented Generation (RAG) optimization tunes the retrieval step in the RAG process so that fewer unnecessary context tokens reach the model. Techniques such as hierarchical chunking, metadata-based filtering, and reranking narrow the retrieved context to the most relevant chunks, so you pay primarily for content that improves the answer.
Two complementary Accenture patterns work alongside these AWS native levers. Context scoping limits the data each agent receives to the document slice relevant to its step of the workflow, rather than passing the full case context at every handoff. Context compression summarizes or filters documents before they reach the model. Both reduce input token volume, especially across multi-agent chains.
Phase 3: Infrastructure and training investment
Semantic caching stores responses and reuses them when a new request is semantically similar to one already answered, even if the wording differs. This avoids redundant model invocations and reduces both cost and latency. On AWS, you can implement it with a vector store such as Amazon ElastiCache in front of the model. In an AWS benchmark on 63,796 real chatbot queries, a 0.90 similarity threshold produced a 74.5 percent cache hit rate, 92.3 percent accuracy, and 72.5 percent cost savings (see ElastiCache semantic caching benchmarks). As you apply routing and semantic caching, keep Amazon Bedrock Guardrails on every route, and set semantic cache expiry so that users don’t receive outdated answers.
Model distillation transfers knowledge from a large teacher model to a smaller, faster student model. Distilled models in Amazon Bedrock are up to 500 percent faster and up to 75 percent less expensive than original models, with less than 2 percent accuracy loss for use cases such as RAG. Distillation is best suited to stable, high-volume workloads where the transaction volume justifies the training investment.
In production: Four client engagements
Accenture applied these levers in four client engagements. Each engagement targeted a different cost pattern. In two of them, Accenture applied several changes together, so those results reflect the combined effect rather than any single lever.
The first engagement was a large enterprise deployment where every request went to a frontier model, with no caching, model tiering, or batching. Accenture applied three changes together: model tiering to match each task to the smallest model that could handle it well, prompt caching for repeated context, and batch processing for overnight and non-urgent jobs. Together they cut monthly token cost by 85 percent, and the saving was not separated by change. About 15 percent of tasks still needed the top model for hard, open-ended reasoning, and the rest ran on lighter models. Your split will depend on your workload composition.
The second engagement was a telecom operator with $12 million in annual token spend. Most of that spend came from agents fanning out requests without limits and from every request going to a frontier model regardless of complexity. Accenture applied triage routing, semantic caching, and context compression together. Input tokens fell by 70 percent, and annual token spend fell from $12 million to $3.8 million, a 68 percent reduction. Accenture validated output quality against the operator’s pre-change performance benchmarks.
The third and fourth engagements each applied a single cache layer. An insurer used model-layer prefix caching, reusing the static portions of its prompts across invocations. A mining operator used an application-layer semantic cache, returning stored responses for repeated queries before they reached the model. Across the two engagements, token spend fell by about 40 percent combined, without model routing changes or training investment.
Results
Outcomes ranged from about 40 percent to about 85 percent, reflecting each client’s starting point. The large enterprise deployment began with no optimization in place and reached 85 percent. The telecom operator reached 68 percent on $12 million in annual token spend, and the insurer and mining operator reached about 40 percent combined with one caching lever each. Results are specific to each client’s workload and may vary.
These savings aren’t the end goal. They create capacity to scale AI workloads, reinvest in higher-value use cases, and improve the return generated by each dollar of AI consumption. Two of these engagements applied several changes at once. When you apply one lever at a time, you can attribute each result to a specific change. AWS and Accenture recommend this sequence for your own workloads: connect consumption to owners and outcomes, apply one lever at a time, measure cost, quality, and business value, and build on what works.
Production governance: Four operating tenets
Optimizing AI economics in production requires an ongoing governance cycle spanning cost, quality, risk, and business value, not a one-time technical change. Before you apply any lever, establish a quantitative baseline for both cost and quality so that you can measure the impact of each change against a known starting point. When you validate output quality, use held-out evaluation sets drawn from your actual production traffic rather than synthetic benchmarks. Apply one lever at a time per workload so that you can attribute results to a specific change and roll back cleanly if quality drops below your acceptable thresholds. If you operate in a regulated industry such as financial services, insurance, or healthcare, document your quality test results and rollback procedures as part of your standard change management record.
These four tenets apply regardless of which levers you are using or what phase of the sequencing you are in. Amazon Bedrock AgentCore is an environment where you can build, connect, and optimize agents at scale with any framework or model. It includes AgentCore Evaluations, a capability of Amazon Bedrock AgentCore that provides batch evaluation to support your quality validation cycle as you progress through more advanced levers.
Getting started
You can begin with three steps. First, turn on cost attribution for Amazon Bedrock by applying cost allocation tags at the team and workload level so that your spend is traceable from day one. Second, activate those tags in the AWS Billing and Cost Management console. Then use CUR 2.0 and model invocation logging to identify high-consumption workloads, and assess each one based on spend, business value, quality requirements, risk, volume, and optimization potential. Third, apply Amazon Bedrock prompt caching to that workload’s interactive traffic and Amazon Bedrock batch inference to its asynchronous jobs as configuration-only changes, and measure the result before you move to architectural changes in Phase 2.
The AWS Well-Architected Agentic AI Lens includes model invocation and token cost optimization (AGENTCOST02), a focus area within the cost optimization pillar for agentic workloads. This focus area provides a structured framework you can use to apply these levers at scale across your organization.
About the authors
Ilan Geller
Ilan is a Managing Director at Accenture, focused on artificial intelligence, delivery, and client growth.
Steven Cox
Steven is an Associate Director in the Accenture AWS Business Group and a Principal Architect specializing in AWS. He leads token economics work across Accenture client engagements.
Manoj Ravulapati
Manoj is a Senior Customer Solutions Manager at AWS who leads strategic AI initiatives, helping enterprises scale agentic AI.
Rajdeep Banerjee
Rajdeep is a Senior Solutions Architect in the AWS Cloud Center of Excellence. He works with AWS partners to design and optimize generative AI workloads on Amazon Bedrock.
正文保留英文原文,标题与摘要由机器翻译。
原文发布于 AWS Machine Learning(Optimize AI economics on Amazon Bedrock with Accenture’s sequenced approach),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。