ArisGlobal 在 Bedrock 上扩展生成式 AI 推理

ArisGlobal 是一家生命科学领域的技术供应商,业务覆盖药物警戒、法规事务、医学事务、临床和质量等职能。该公司开发了 NavaX——一个基于 Amazon Bedrock、专为生命科学打造的 AI 引擎,用于自动化处理不良事件个案。ArisGlobal 选择 Amazon Bedrock,是看重其托管式基础模型访问、欧盟地理范围内的跨区域推理,以及托管式提示缓存。随着企业客户提出各自不同的合规要求,共享提示前缀在 6 个月内增长了约 150%,而单个案件的推理成本上升超过 200%。
ArisGlobal 最初缓存每个源文档,因为它是请求中占比最大的部分。这样一来,不断增长的共享提示前缀就落在缓存之外。通过颠倒顺序、改为缓存共享前缀,ArisGlobal 降低了实际输入成本,并在不带来成本同比例增长的情况下支持了更多租户。
ArisGlobal 是 LifeSphere 的开发者。LifeSphere 是一个统一 SaaS 平台,面向药物警戒、法规事务、质量和医学信息。LifeSphere 服务全球制药企业,其中包括多家按营收排名前 20 的制药机构(来源:ArisGlobal 新闻稿)。整个系统运行在 AWS 上。
LifeSphere NavaX 是 ArisGlobal 的 AI 引擎,它将药物安全从记录系统转变为战略优势系统。2025 年 3 月,ArisGlobal 携手 Anthropic 的 Claude,在 Amazon Bedrock 上将其首个生成式 AI 解决方案投入生产。根据 ArisGlobal 的内部生产遥测数据,他们已在受监管的制药环境中处理超过 100 万笔真实交易,跨多个企业租户处理了数十亿 Token。模型可用性因区域而异。参见 AWS 区域模型支持情况。
当制药公司收到不良事件报告时,通常是一份描述患者对某种药物反应的多页 PDF,NavaX 的 Dynamic Data Extraction (DDE) 智能体会提取约 130 个字段。这些字段包括患者人口统计信息、可疑药物、报告者信息、事件时间线、严重性标准和监管分类。由于这些内容包含受保护的健康信息,ArisGlobal 将负责任 AI 控制措施作为其生产部署的标准组成部分。它利用 Amazon Bedrock Guardrails 等能力来过滤内容,并检测或掩蔽敏感信息。提取的字段还会根据 schema 进行验证,并以源文档为依据。
这种提取通过向模型发出的多个顺序请求完成。每个请求聚焦于病例表单的不同部分,并且每个请求都携带:
两种输入在不同范围内重复。共享的提示前缀包括系统规则、提取 schema 和少样本示例。它会在使用同一智能体的请求、病例和租户之间重复。源文档仅会在与一个病例相关的请求中重复。
NavaX 在首个企业租户上线时,系统提示还很精简。随着 ArisGlobal 接入更多制药公司,每家都带来了特定需求:
在 6 个月里,共享提示前缀从约 72,000 个 Token 增长到 180,000 个 Token,增幅达 150%。由于每个病例有多个顺序请求,单病例推理成本增加了 200% 以上。
该前缀会随着每个租户增长,并在每次请求时被重新处理,因此提示的增长会随交易量而叠加。
表 1 比较了最初的文档优先请求顺序与生产环境中的提示优先设计。
Amazon Bedrock 提示词缓存通过缓存从请求开头到缓存检查点之间连续的提示词前缀,降低延迟和输入 Token 成本。后续具有相同前缀的请求可以按折扣费率读取已缓存的 Token,而无需再次处理它们。缓存条目设有生存时间(TTL),在成功命中缓存后重置。
缓存基于前缀而非基于块。请求各部分按顺序评估——工具、系统,然后是消息——因此更改靠前的部分会使紧随其后的匹配内容失效。
借助显式缓存检查点,你可以选择确切的前缀边界。受支持的 Claude 模型允许每个请求最多设置四个检查点。
借助简化的缓存管理,你只需在静态内容之后放置一个检查点。Amazon Bedrock 会从该点开始向后搜索最长的匹配前缀,最多约 20 个内容块。
ArisGlobal 在共享提示词前缀之后立即使用一个显式检查点,该前缀最先发送。每个案例的文档位于该检查点之后,并在每次请求时被处理。
这颠倒了最初的请求顺序。由于只有前缀可以被缓存,因此顺序决定了哪些内容可以被复用。
第一种设计将 10–200 页的源文档放在每个请求的开头,并将检查点置于其后。共享提示词前缀和针对特定章节的问题则位于检查点之后。
{ "modelId": "anthropic.claude-sonnet-4-5-20250929-v1:0", "messages": [ { "role": "user", "content": [ { "text": "[完整的不良事件 PDF 内容]..." // <-- 文档最先发送 }, { "cachePoint": {"type": "default"} // <-- 紧跟文档之后的检查点 }, { "text": "[系统指令 + 合规规则 + schema + few-shot 示例 + 客户专属提示词]..." // <-- 检查点之后的部分:永不缓存 }, { "text": "提取第 3 节的字段:疑似药物信息" } ] } ] } 这套设计在单个案例内部复用了文档:第一个请求将其写入缓存,之后针对该案例的请求则以折扣费率读取。
下一个案例使用了不同的文档,需要重新写入缓存,因此复用从未跨案例或跨租户延伸。
更重要的是,共享的提示词前缀仍位于检查点之后,因而在每一次请求中都以全额输入费率被处理。真正驱动成本曲线的那些内容,被排除在缓存之外。
生产环境的设计改变的是顺序,而不是再增加一个检查点:共享的提示词前缀放在最前面,检查点紧随其后,文档则放在最后。
如今,同一个检查点创建出一个可在全机群范围内复用的缓存条目:
ArisGlobal 放弃了单个案例内有限的文档复用,换来了跨整个机群的提示词复用。而共享前缀恰恰正是随客户增加而不断增长的那一部分。
{ "modelId": "anthropic.claude-sonnet-4-5-20250929-v1:0", "system": [ { "text": "[共享系统指令 + 合规规则 + 基础抽取 schema + few-shot 示例]..." }, { "cachePoint": {"type": "default"} // <-- 稳定提示词之后的检查点 } ], "messages": [ { "role": "user", "content": [ { "text": "[完整的不良事件 PDF 内容——位于检查点之后,不缓存]..." }, { "text": "抽取第 5 节的字段:事件评估" // <-- 只有这一处会随每次请求变化 } ] } ] } 对大多数活跃账户而言,每个租户大约每 5 分钟就有一个病例到达,并且有多个租户并发运行。这种流量在营业时间内让共享前缀保持热状态,因为每次缓存命中都会重置 TTL。
若在文档之后再加第二个检查点,就会缓存一个由共享提示词与该病例文档共同组成的前缀。该条目仍然只对单个病例唯一。
因此,ArisGlobal 优先选择了读取量可在整个集群中摊薄的共享提示词前缀。共享提示词前缀可跨病例、跨租户复用,而每个病例的文档则不然。
在生产环境中,典型请求有 65% 的输入 Token 由共享提示词缓存提供。图 1 显示,缓存读取量持续超过缓存写入量。
图 1:Amazon CloudWatch 提示词缓存使用情况。在生产流量时段,蓝色的缓存读取曲线升至约缓存写入曲线的 10 倍。
在欧盟地理跨区域推理配置(cross-Region inference profile)保持不变的情况下,缓存共享提示词前缀带来了以下结果:
图 2:端到端调用延迟。该图显示,采用提示词缓存后,延迟区间更低且更稳定。
图 3:调用次数。该图显示生产请求量保持稳定,而推理开销的增长并未与请求量同比例上升。
根据 ArisGlobal 的生产计费数据,65% 的输入 Token 由缓存提供,使有效输入成本降低了约 50%。在每月超过 190 亿 Token 的规模下,这带来了约 40% 的月度推理支出整体节省。
注:本节中所有性能与成本数据均反映 ArisGlobal 向 AWS 报告的生产指标。实际结果会因提示词结构、流量模式和缓存命中率的不同而有所差异。
ArisGlobal 的经验为许多具有大型、重复提示词前缀的工作负载提供了参考:
缓存重复出现的内容,而不是体积最大的内容。单个案例的大型文档可复用性有限,而字节级完全一致的共享提示词前缀则可为整个集群的请求提供服务。
将稳定内容放在最前面。把工具定义、系统指令、schema 和示例排在检查点之前。将租户标识符、案例数据和单次请求的问题放在其后。
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(How ArisGlobal scaled generative AI inference on Amazon Bedrock)
ArisGlobal, a provider of life sciences technology spanning pharmacovigilance, regulatory, medical affairs, clinical, and quality functions, developed NavaX, an AI engine purpose-built for life sciences on Amazon Bedrock, to automate adverse-event case processing. ArisGlobal chose Amazon Bedrock for managed access to foundation models, EU geographic cross-Region inference, and managed prompt caching. As enterprise customers added distinct compliance requirements, the shared prompt prefix grew by about 150% over 6 months, while per-case inference costs increased by more than 200%.
ArisGlobal first cached each source document because it was the largest part of a request. That left the growing shared prompt prefix outside the cache. By reversing the order and caching the shared prefix instead, ArisGlobal reduced effective input cost and supported additional tenants without proportional cost growth.
Who is ArisGlobal?
ArisGlobal is the creator of LifeSphere, a unified SaaS platform for pharmacovigilance, regulatory affairs, quality, and medical information. LifeSphere serves pharmaceutical companies globally, including multiple Top 20 pharma organizations by revenue (source: ArisGlobal press release). The entire system runs on AWS.
LifeSphere NavaX is ArisGlobal’s AI engine that transforms drug safety from a system of record into a system of strategic advantage. ArisGlobal productionized their first generative AI solution on Amazon Bedrock with Anthropic’s Claude in March 2025. According to ArisGlobal’s internal production telemetry, they have processed over 1 million real transactions in regulated pharmaceutical environments, handling billions of tokens across multiple enterprise tenants. Model availability varies by Region. See Model support by AWS Region.
How NavaX processes an adverse event case
When a pharmaceutical company receives an adverse-event report, typically a multipage PDF describing a patient’s reaction to a medication, NavaX’s Dynamic Data Extraction (DDE) agent extracts around 130 fields. These fields include patient demographics, suspect drugs, reporter information, event timelines, seriousness criteria, and regulatory classifications. Because this content includes protected health information, ArisGlobal applies responsible-AI controls as a standard part of its production deployment. It draws on capabilities such as Amazon Bedrock Guardrails to filter content and detect or mask sensitive information. Extracted fields are also validated against the schema and grounded in the source document.
This extraction happens across multiple sequential requests to the model. Each request focuses on a different section of the case form, and each carries:
- System instructions — model behavior rules, output format specifications, and regulatory compliance constraints.
- Customer-specific rules — per-tenant extraction logic, field mappings, and business rules that differ by pharmaceutical company.
- Extraction schema — around 130 target fields with types, validation rules, and few-shot examples.
- The source document — the adverse event PDF (often 10–200 pages).
Two inputs repeat at different scopes. The shared prompt prefix includes the system rules, extraction schema, and few-shot examples. It repeats across requests, cases, and tenants using the same agent. The source document repeats only for the requests associated with one case.
The prompt bloat problem
When NavaX launched with its first enterprise tenant, the system prompt was lean. As ArisGlobal onboarded additional pharmaceutical companies, each brought specific requirements:
- Custom field mappings (Tenant A maps reaction onset differently from Tenant B).
- Regulatory jurisdiction rules (EU ICSR compared with FDA MedWatch and PMDA formatting).
- Company-specific business logic for seriousness assessment and causality.
- Per-tenant few-shot examples calibrated to each company’s historical cases.
Over 6 months, the shared prompt prefix grew from approximately 72,000 to 180,000 tokens, a 150% increase. With multiple sequential requests per case, per-case inference costs increased by more than 200%.
The prefix grew with every tenant and was reprocessed on every request, so prompt growth compounded with transaction volume.
Table 1 compares the original document-first request order with the production prompt-first design.
| Aspect | First approach | Production design |
| Request order | Per-case document, cache checkpoint, shared prompt prefix | Shared prompt prefix, cache checkpoint, per-case document |
| Cached prefix | The document, which changes for every case | The shared prompt prefix, which repeats across requests |
| Reuse scope | Only the remaining requests for one case | Requests across cases and tenants using the same agent |
| Cost effect | The growing prompt is repeatedly processed at the full input rate | Fleet-wide prompt reuse supports sublinear cost growth |
| EU routing | EU geographic cross-Region inference remains unchanged | EU geographic cross-Region inference remains unchanged |
Amazon Bedrock prompt caching strategies
Amazon Bedrock prompt caching reduces latency and input token costs by caching a contiguous prompt prefix from the start of a request through a cache checkpoint. A later request with the same prefix can read cached tokens at a reduced rate instead of processing them again. Cache entries have a time-to-live (TTL) that resets after a successful cache hit.
Caching is prefix-based rather than block-based. Request sections are evaluated in sequence—tools, system, then messages—so changing an earlier section invalidates matching content that follows it.
Cache checkpoints (explicit control)
With explicit cache checkpoints, you can choose the exact prefix boundary. Supported Claude models allow up to four checkpoints per request.
- Place each checkpoint after a stable, reusable prefix.
- Meet the model-specific minimum prefix size. For Claude Sonnet 4.5 the minimum is 1,024 tokens (see supported models and Regions).
- Checkpoints are evaluated in this order: tools, system, and messages.
- Changing an earlier section invalidates cache matches for later sections.
- The default TTL is 5 minutes. Select models support a 1-hour TTL by setting
"ttl": "1h". Longer-TTL entries must precede shorter-TTL entries.
Simplified cache management (automated)
With simplified cache management, you place one checkpoint after static content. Amazon Bedrock searches backward from that point for the longest matching prefix, up to about 20 content blocks.
- One checkpoint can locate the longest matching prefix automatically.
- Use explicit checkpoints when stable content extends beyond the lookback range.
What ArisGlobal cached, and what they deliberately did not
ArisGlobal uses one explicit checkpoint immediately after the shared prompt prefix, which is sent first. The per-case document follows the checkpoint and is processed on each request.
This reverses the initial request order. Because only a prefix can be cached, ordering determines which content can be reused.
The launch design: Prioritizing document reuse
The first design put the 10–200 page source document at the start of each request and placed the checkpoint after it. The shared prompt prefix and section-specific question followed the checkpoint.
{
"modelId": "anthropic.claude-sonnet-4-5-20250929-v1:0",
"messages": [
{
"role": "user",
"content": [
{
"text": "[Full adverse event PDF content]..." // <-- Document sent FIRST
},
{
"cachePoint": {"type": "default"} // <-- Checkpoint right after the document
},
{
"text": "[System instructions + compliance rules + schema +
few-shot examples + customer-specific prompt]..."
// <-- After the checkpoint: never cached
},
{
"text": "Extract fields for Section 3: Suspect Drug Information"
}
]
}
]
}
The design reused the document within one case: the first request wrote it to cache, and later requests for that case read it at the reduced rate.
The next case used a different document and required a new cache write, so reuse never extended across cases or tenants.
More importantly, the shared prompt prefix remained after the checkpoint and was processed at the full input rate on every request. The content driving the cost curve was excluded from the cache.
The evolved production design: Prioritizing shared-prefix reuse
The production design changed the order rather than adding another checkpoint: the shared prompt prefix comes first, the checkpoint follows it, and the document comes last.
The same checkpoint now creates a reusable fleet-wide cache entry:
- Before — the cached document prefix was unique to one case and the shared prompt was never cached.
- After — the cached shared prompt prefix repeats across requests, cases, and tenants using the same agent.
ArisGlobal gave up limited document reuse within one case and gained prompt reuse across the fleet. The shared prefix is also the part that grows as customers are added.
Updated implementation
{
"modelId": "anthropic.claude-sonnet-4-5-20250929-v1:0",
"system": [
{
"text": "[Shared system instructions + compliance rules +
base extraction schema + few-shot examples]..."
},
{
"cachePoint": {"type": "default"} // <-- Checkpoint after the stable prompt
}
],
"messages": [
{
"role": "user",
"content": [
{
"text": "[Full adverse event PDF content — sits after the
checkpoint, not cached]..."
},
{
"text": "Extract fields for Section 5: Event Assessment"
// <-- Only this changes per request
}
]
}
]
}
For most active accounts, cases arrive about every 5 minutes per tenant and multiple tenants run concurrently. This traffic keeps the shared prefix warm during business hours because each cache hit resets the TTL.
Caching both the prompt and the document
A second checkpoint after the document would cache a combined prefix consisting of the shared prompt and that case’s document. The entry would still be unique to one case.
ArisGlobal therefore prioritized the prefix whose reads amortize across the fleet. The shared prompt prefix scales across cases and tenants. The per-case document does not.
In production, 65% of input tokens on a typical request are served from the shared prompt cache. Figure 1 shows cache reads consistently exceeding cache writes.
Figure 1: Amazon CloudWatch prompt-cache usage. The blue cache-read series rises to roughly 10 times the cache-write series during periods of production traffic
Results
Caching the shared prompt prefix produced the following results while the EU geographic cross-Region inference profile remained in place:
- 65% of input tokens served from cache at the reduced read rate.
- Up to 25% lower end-to-end latency on cached production requests.
- Approximately 50% lower effective input cost.
- Approximately 40% lower overall monthly inference spend at more than 19 billion tokens per month.
- Around 15% more enterprise tenants onboarded without proportional cost increases.
- EU geographic routing unchanged. Prompt caching operates within the existing inference configuration.
Figure 2: End-to-end invocation latency. The chart shows a lower and more stable latency range after prompt-cache adoption
Figure 3: Invocation count. The chart shows sustained production request volume while inference spend grew less than proportionally
Based on ArisGlobal’s production billing data, serving 65% of input tokens from cache reduced effective input cost by approximately 50%. At more than 19 billion tokens per month, this produced approximately 40% overall monthly savings in inference spend.
Note: All performance and cost figures in this section reflect ArisGlobal’s production metrics as reported to AWS. Results will vary based on prompt structure, traffic patterns, and cache hit rates.
Production learnings
ArisGlobal’s experience offers guidance for many workloads with large, repeated prompt prefixes:
Cache what repeats, not what is largest. A large per-case document offers limited reuse, while a byte-identical shared prompt prefix can serve requests across the fleet.
Put stable content first. Order tool definitions, system instructions, schemas, and examples before the checkpoint. Place tenant identifiers, case data, and the per-request question afterward.
Use fleet traffic to amortize cache writes. Requests from tenants using the same agent can read the same prefix, and cache hits reset the TTL. For sporadic traffic, evaluate the 1-hour TTL on supported models.
Reduce avoidable calls. Each call still processes the uncached document, so bundle sequential questions when output limits and schema requirements permit.
Keep the prefix byte-identical. Timestamps, request IDs, tenant names, and reordered tool definitions can cause misses. Move every dynamic element after the checkpoint.
Monitor the production baseline. Track cache reads, cache writes, and total input tokens in CloudWatch. Alert on a sustained drop below the normal 65% cache-hit baseline rather than using an incompatible 80% threshold.
Check model-specific limits. Prefix minimums and TTL options vary by model. Claude Sonnet 4.5 requires at least 1,024 tokens for a cache checkpoint.
Conclusion
ArisGlobal combined EU geographic cross-Region inference with prompt caching aimed at the right content. Caching the stable shared prefix instead of the per-case document reduced effective input cost by approximately 50%, contributed to approximately 40% overall monthly inference savings, and supported around 15% more enterprise tenants without proportional cost increases.
The key lesson is that cache value depends on reuse, not size. For multi-tenant agentic workloads, a repeated shared prompt prefix can be more valuable than a larger document that changes for every case.
To learn more, see the Amazon Bedrock prompt caching documentation, the cross-Region inference guide, and the AWS blog post Effectively use prompt caching on Amazon Bedrock.
About the authors
Hitesh Sharma
Hitesh is a technology leader with 25 years of experience in software engineering and product innovation, including 20 years in the Life Sciences industry. He specializes in applying Generative AI, AI/ML, and cloud-native technologies to solve complex challenges across Pharmacovigilance, Drug Safety, regulatory compliance, and patient safety. Hitesh has led the development of scalable AI platforms that deliver business value. He is passionate about GenAI, innovation, and building high-performing teams to advance Life Sciences technology.
Rakesh Kumar Sharma
Rakesh Kumar Sharma is an AI/ML leader and researcher with deep expertise in Generative AI, Agentic AI, and enterprise AI architecture. He specializes in designing and deploying scalable AI solutions using LLMs, AI agents, RAG, and modern ML frameworks, enabling intelligent automation, knowledge-driven decision-making, and production-ready AI platforms. His work focuses on architecting agentic systems that combine reasoning, retrieval, tools, and autonomous workflows to solve complex enterprise challenges.
Shubham Gupta
Shubham is a Solutions Architect at AWS based in Bengaluru, India, supporting independent software vendors (ISVs). He works with engineering and leadership teams to design, build, and run their products on AWS, from first architecture to production, with a deep focus on generative AI and resilience at scale. Outside of work, Shubham is an avid hiker who brings the same preparation and persistence from the trail to building systems that last.
Vibhu Pareek
Vibhu is a Sr. Solutions Architect at AWS. Since 2016, he has guided customers in cloud adoption using well-architected, repeatable patterns. With his specialization in databases, data analytics, and AI, he thrives on transforming complex challenges into innovative solutions. Outside work, he enjoys short treks and sports like badminton, football, and swimming.
正文保留英文原文,标题与摘要由机器翻译。
原文发布于 AWS Machine Learning(How ArisGlobal scaled generative AI inference on Amazon Bedrock),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。