如何用三层架构大规模监管 AI 智能体

借助带外监督器实现规模化 AI 智能体监控
我与大型企业的许多数据与 AI 负责人交流过,了解他们的智能体部署进展如何、如何组织团队,以及他们如何看待可靠性与信任。
我注意到,大多数组织在设计智能体架构时都采用双层结构。底层是一组执行领域任务的工作智能体,例如查询内部数据库、回答客户咨询或分派工单。紧接其上的则是人类工程师和运营人员,他们负责审查智能体的输出、管理审批、处理异常并统筹整个系统。
这套模式在业界已有大量记录。例如,麦肯锡描述过由 2 到 5 人监督 50 到 100 个专用智能体、共同运行端到端流程的团队。微软则提出了“agent boss”这一说法,认为每位员工最终都会成为“agent boss”,而人与智能体的比例将成为一项标志性的业务指标。
尽管这在概念上说得通,但在实践中,这种双层架构存在一个明显的缺口。
人的速度与机器的速度之间存在着无法调和的差异。智能体每小时要做出成千上万个决策,以任何人都不可能跟得上的速度调用 API、运行 SQL 查询。而一名人类工程师解析单条调用链就需要几分钟。对于一个管理 5 个智能体的团队来说,维持人工审查或许还可行,但试想当一个组织扩展到 50 个、100 个甚至数千个智能体时会怎样。此时,人在回路队列不再是防护措施,反而会变成工程团队难以承受的积压。
因此,要可靠地扩展 AI,就必须把实时运营监管与人工审核分离开来。为此,我们必须把两层模型扩展为独立的三层架构。这需要在工作智能体与治理它们的人类之间,插入一个带外的、自主的监督层。
瓶颈:为何同步的人在回路模式会失效
Monte Carlo 与 CDO Magazine 在 2026 年初联合开展了一项《AI 可靠性现状》调查,就 AI 规模化的看法与规划,对数据与 AI 高管进行了调研。在该研究中,58% 的领导者表示,他们预计未来 12 个月内其智能体数量将大幅增加。然而,62% 的受访者表示依赖人工审核来验证 AI 输出,仅 25% 使用自动化监控。
这种脱节为风险敞开了大门。在依赖人工审核的同时大规模部署智能体,会使可靠性问题发生与被发现、被解决之间出现滞后——而这种滞后只会随时间不断累积,最终阻碍 AI 的安全部署。
数据表明,这种情况已经在发生,只是规模较小:在我们的研究中,61% 的组织表示遭遇过静默故障,即被监控指标看似正常、而严重问题却正在发生的情形。
在前面提到的同一份 McKinsey 报告中,他们得出了类似结论,指出“智能体式 AI 的采用规模将受制于人类能提供多少监督能力——这使治理本身成为生产力的潜在瓶颈。”
人类监督在规模化时难以跟上节奏,原因有若干。首先是纯粹的操作带宽限制。一名工程师现实中最多只能监控几个活跃智能体和少量数据管道,再往后审查质量就会下降。
另一个因素是上下文延迟。当智能体出现异常行为时,要沿着动态规划路径、向量检索分块以及底层 schema 变更一路追溯根因,需要数小时的排查与故障定位,而这根本跟不上不断增长的问题队列。
最后,还有典型的智能体失效模式:静默漂移。智能体的失败很少会以抛出明显异常的方式出现,让团队可以迅速定位。相反,它们会基于有缺陷的中间工具调用或过时的上游数据,产出看似自信、看似有效的输出,并在系统中传播。人类操作员在扫视告警或查看仪表盘时,并不容易发现这些问题。
三层运营架构
我认为,要在规模化条件下保障智能体表现,应当采用一种监督式模型,而不是持续让人类承担智能体审查工作。该模型把运营层面的排查工作交给一个自动化的带外层,由人类负责治理策略、审批关键的高影响操作,而自动化监督者则负责监管运行时健康状况与系统数据。
三层监督式架构。图片由作者提供。
主要交互与产物
策略定义、风险参数化、审计审查以及高影响操作的审批。
审阅带预拟补救上下文的优先升级事项;批准或拒绝 schema 和 Prompt 更新。
第二层:带外监督器
持续链路追踪评估、仓库数据校验、异常关联、可逆遏制以及补救方案起草。
以带外方式摄取网关/代理链路数据、仓库表和血缘信息;下发有边界的任务级动作。
第一层:生产环境智能体
领域任务执行(SQL 生成、客户工作流、文档解析、API 调用)。
执行工具调用;输出结构化遥测数据;运行时对第二层无感知。
在这一架构中,人审阅的是「审阅者」,而非解读原始工作内容。这意味着评估由监督系统升级上报、经过优先级排序的决策。
独立监督的准则
业界专家也预见到这一层的到来。
例如,Gartner 将监督者技术归类为“守护型智能体”,并在 2025 年 6 月预测,到 2030 年,守护型智能体将占据智能体式 AI 市场至少 10% 至 15% 的份额。
麦肯锡同样预计,各组织会将“控制智能体”嵌入工作流,由“批判”智能体对输出提出质疑,由“防护机制”智能体执行策略。
然而,我的观点不同之处在于,如今大多数工具构建这一操作者的方式:以内联防护机制的形式。这带来了明显的冲突;如果监督层以带内方式运行,它就会继承其本应检查的系统的运行时漏洞、超时限制和故障模式。
因此,有效的监督层必须遵循某些结构性设计规则:
运行时与凭证分离:监督者运行在专用算力上,并使用独立的服务账号。工作智能体无法读取、修改或终止监督者进程。
它绝不接受受其监督的智能体下达的指令:工作智能体的输出被视为需要分析的遥测数据,而绝不是可执行命令。
它的证据来自工作智能体无法写入的来源:监督器不依赖工作智能体自我报告的指标。它摄取独立的遥测数据:由智能体无法控制的网络网关或 LLM 代理捕获的追踪数据、仓库查询日志以及底层表指标。
不可变审计追踪:每一次观察、修复尝试和升级决策都会提交到人类工程师可访问的仅追加审计日志中。
实施任务级自主边界
监督层不能以二元的“可信 vs. 不可信”权限集来运作。我认为自主性必须在任务级建立,关注单个操作可能造成多大损害,而不是适用于整个智能体。
自动化监督器应当能够自主执行非破坏性、可逆的操作。另一方面,不可逆、高影响的变更应当升级给人类操作员。
为了更具体地说明,请看下表,它展示了一个发票对账流水线的示例策略。人类提前编写策略,围绕风险等级和升级指令设置清晰的参数。
监督器,自动
使用回退费率的转换占比超过 15%,或追踪数据看起来异常
监督器自动处理
上游数据延迟超过 90 分钟
需人工审批;若 30 分钟内无人响应,监督器改为暂停该批次
检测到上游 schema 变更
回滚系统提示词
有据性得分低于 0.70
在实践中,监督器的逻辑很简单:对照独立计算出的指标逐条评估规则,可直接执行的动作自动执行,需要审批的动作则转交给人工操作员,并附带截止时间,以及无人响应时的安全回退方案。
由于这套策略是以配置的形式写下来的,而不是埋在代码里,它可以像任何其他配置一样被审查、做版本管理和审计,并且可以随着对监督器信任度的提升而收紧或放宽。
带外事件解决流程是什么样的
让我们通过一个示例,看看事件解决在这种监督架构下是如何展开的。
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(How to Oversee AI Agents at Scale With a Three-Tier Architecture)
AI Agent Monitoring at Scale With an Out-of-Band Supervisor
I talk to a lot of data and AI leaders at large enterprises about how their agentic deployments are going, how they are organizing their teams, and how they are thinking about reliability and trust.
Most orgs, I have noticed, are designing agent architectures with a two-layer structure. At the base sits a group of working agents executing domain tasks, such as querying internal databases, answering customer inquiries, or routing tickets. Directly above them sit the human engineers and operators who review their outputs, manage approvals, resolve anomalies, and orchestrate the entire system.
This set-up is well documented across the industry. McKinsey, for example, describes teams of two to five people supervising 50 to 100 specialized agents that run an end-to-end process. Microsoft, meanwhile, coined the term “agent boss,” saying every employee will eventually become one, and that the ratio of humans to agents will emerge as a defining business metric.
While this conceptually makes sense, there is a significant gap in this two-layer architecture in practice.
There is an irreconcilable difference between human speed and machine speed. Agents make thousands of decisions every hour, calling APIs and running SQL queries at a rate that no person could ever keep track of. A human engineer, meanwhile, requires minutes to parse a single trace. For a team managing five agents, maintaining human review might be possible, but imagine when an organization scales to fifty, one hundred, or even thousands of agents. The human-in-the-loop queue ceases to be a safeguard and becomes an overwhelming backlog for engineering teams.
Scaling AI reliably, therefore, requires separating real-time operational oversight from human review. To do this, we have to expand the two-tier model into an independent three-tier architecture. This requires inserting an out-of-band, autonomous supervisory layer between the working agents and the humans who govern them.
The bottleneck: why synchronous human-in-the-loop fails
Monte Carlo conducted a State of AI Reliability survey with CDO Magazine in early 2026, polling data and AI executives about their perspectives and plans on scaling AI. In this study, 58% of leaders reported that they expect their agent counts to increase substantially over the next 12 months. However, 62% of respondents reported relying on manual human review to verify AI outputs, with only 25% using automated monitoring.
This disconnect leaves the door open for risk. Deploying agents at scale while relying on human review creates a lag between when reliability issues occur and when they are noticed and resolved – a lag that only compounds over time, ultimately preventing safe AI deployment.
The data suggests that this is already happening, albeit on a smaller scale: 61% of organizations from our research reported experiencing silent failures, defined as situations where monitored metrics appeared normal while a critical issue was actively occurring.
In the same McKinsey report mentioned earlier, they reach a similar conclusion, noting that “the scale of agentic adoption will be capped by how much oversight capacity humans can provide—making governance itself a potential bottleneck to productivity.”
Human oversight struggles to keep pace at scale for a number of reasons. First of all, there is the sheer operational bandwidth limitation. An engineer can realistically monitor a couple of active agents and a handful of data pipelines before review quality degrades.
Another factor is context latency. When an agent misbehaves, tracing the root cause back through dynamic planning paths, vector retrieval chunks, and underlying schema changes takes hours of triage and troubleshooting that can’t keep pace with the ever-growing queue of issues.
Lastly, there’s the typical agent failure mode: silent drift.Agents rarely fail in ways that throw obvious exceptions, which teams can quickly pinpoint. Instead, they produce confident, valid-looking outputs based on flawed intermediate tool calls or stale upstream data that propagate through the system. These are not easily picked up by human operators scanning through alerts or reading dashboards.
The three-tier operational architecture
I would argue for a supervisory model for securing agent performance at scale rather than keeping humans assigned to agent reviews. This model delegates operational triage to an automated, out-of-band layer, with humans governing policy and approving critical, high-impact actions, while the automated supervisor oversees runtime health and system data.

The three-tier supervisory architecture. Image by author.
Primary interaction and artifacts
Policy definition, risk parameterization, audit reviews, and high-impact approvals.
Reviews prioritized escalations with pre-drafted remediation context; approves or rejects schema and prompt updates.
Tier 2: Out-of-band supervisor
Continuous trace evaluation, warehouse data checks, anomaly correlation, reversible containment, and remediation drafting.
Ingests gateway/proxy traces, warehouse tables, and lineage out-of-band; dispatches bounded, task-level actions.
Tier 1: Production agents
Domain task execution (SQL generation, customer workflows, document parsing, API calls).
Executes tool calls; emits structured telemetry; operates without execution awareness of Tier 2.
In this architecture, people review the reviewer rather than interpreting the raw work. This means evaluating prioritized decisions that are escalated by the supervisory system.
Tenets of independent supervision
Industry experts also see this layer coming.
Gartner, for example, categorizes supervisor technology as “guardian agents,” predicting in June 2025 that guardian agents will capture at least 10% to 15% of the agentic AI market by 2030.
McKinsey similarly anticipates organizations embedding “control agents” into workflows, with “critic” agents challenging outputs and “guardrail” agents enforcing policy.
Where my perspective differs, however, is in how most tools build this operator today: as inline guardrails. This presents a clear conflict; if a supervisory layer runs in-band, it inherits the runtime vulnerabilities, timeout limits, and failure modes of the system it is intended to check.
An effective supervisory layer, therefore, must adhere to certain structural design rules:
Separate runtime and credentials: The supervisor operates on dedicated compute and separate service accounts. The working agents can’t read, modify, or terminate the supervisor process.
It never takes instructions from the agents it oversees: Working agent outputs are treated as telemetry to be analyzed, never as executable commands.
Its evidence comes from sources the working agents can’t write to: The supervisor does not rely on self-reported metrics from working agents. It ingests independent telemetry: traces captured by a network gateway or LLM proxy the agent does not control, warehouse query logs, and underlying table metrics.
Immutable audit trails: Every observation, remediation attempt, and escalation decision is committed to an append-only audit log accessible to human engineers.
Implementing Task-Level Autonomy Boundaries
A supervisory layer can’t operate with a binary “trusted vs. untrusted” permission set. I would argue that autonomy must be established at the task level, focusing on how much damage a single action can do, rather than applying to the agent as a whole.
An automated supervisor should be able to execute non-destructive, reversible actions autonomously. Irreversible, high-impact changes, on the other hand, should be escalated to human operators.
To make this more tangible, take a look at the table below, which shows an example policy for an invoice reconciliation pipeline. Humans write the policy in advance, setting clear parameters around risk level and escalation instructions.
Supervisor, automatically
Share of conversions using a fallback rate exceeds 15%, or traces look anomalous
Supervisor, automatically
Upstream data is more than 90 minutes late
Human approval; if no response in 30 minutes, the supervisor pauses the batch instead
An upstream schema change is detected
Roll back a system prompt
Groundedness score drops below 0.70
In practice, the supervisor’s logic is simple: evaluate each rule against independently computed metrics, execute autonomous actions directly, and route approval-required actions to a human operator with a deadline and a safe fallback if no one responds.
Because the policy is written down as configuration rather than buried in code, it can be reviewed, versioned, and audited like any other configuration, and tightened or relaxed as trust in the supervisor grows.
What an out-of-band incident resolution looks like
Let’s take a look at how incident resolution plays out with this supervisory architecture by walking through an example.
Consider an agent at a financial institution whose function is to convert currencies on international invoices. The agent relies on an upstream exchange-rate API and an internal data warehouse view.
One day, the third-party exchange-rate API updates its payload schema, renaming the USD exchange rate value. The currency exchange rate agent does not recognize the change. Rather than erroring, however, its fallback logic defaults missing rates to a static 1.00 and continues to process invoices with a 200 status code and converted amounts that look superficially valid, despite being mathematically wrong.
The agent’s self-reported span metrics would not indicate there is a problem, and relying on that telemetry would mean human operators don’t notice the issue until it’s picked up somewhere further downstream after it’s already impacted business users or customers.
With an independent supervisory layer, the reviewer audits the destination warehouse table, not the currency agent’s self-reported telemetry. It calculates that the fallback rate ratio, the share of converted transactions written with an exact rate of 1.00, jumped from an expected baseline of 0.2% to, let’s say, 18.4%, even though the currency agent reported its tool calls as successful.
Checking the policy, the supervisor finds that pausing a batch is considered low risk and is marked autonomous – meaning it can act on this right away. It therefore dispatches a hold request to the queue manager, buffering outgoing invoices before they leave the enterprise perimeter. The supervisor then gets to work on remediation; it inspects proxy payload diffs, detects the renamed API key, drafts a patch for the agent’s mapping configuration, and routes the incident to the on-call engineer’s queue.
In the last phase of this resolution, the engineer receives an alert detailing the incident, including the root cause, affected invoice IDs, held batch status, and a proposed schema patch. The engineer approves it and moves on.
A supervisory layer, despite being separate from functional agents, is software at the end of the day, too. That means it can fail. So do we then need another layer to monitor this operator? What’s to stop the system from regressing into an endless Russian Doll dilemma of agents watching agents, who are watching other agents?
This is a fair question, and it’s why I would recommend keeping supervisory layers much simpler than the agents they monitor. An operator at this level should check rules against the data it gathers itself, but not reason its way to new conclusions. On its own, it should be able to do things that are easy to undo, like pausing a batch or refreshing a table. If something goes wrong, the damage is minimal and contained to time lost rather than revenue or reputation. The decisions it makes should also be logged and consistently spot-checked by human operators, reallocating human attention to the most critical decisions and mitigating risk.
What this means for teams scaling agents today
Scaling AI in the enterprise is more of an operational challenge than a model intelligence problem. Human speed will never be able to keep pace with machines, and so relying purely on teams of people to maintain quality and reliability will lead to engineering fatigue, unmonitored silent drift, and cascading downstream errors.
So where do we start today? A good first step is to take an audit. Examine every action your agents can take and determine how much damage each could do in your organization if they failed. Those you would consider low-risk, easy-to-undo actions are exactly what an automated supervisory layer can monitor and resolve on its own. Save your human team’s time, energy, and brilliance for the high-stakes work. They will surely thank you.
Disclosure: I’m the co-founder and CEO of Monte Carlo.
A practical walkthrough using text-to-SQL as the example
June 24, 202611 min read
A hands-on guide to defining specialist agents and coordinating their work in the Codex CLI
August 28, 20265 min read
How Knives Out teaches Bayesian thinking (without you realizing it)
May 31, 202610 min read
Using a controlled workflow, unique data & prompt chaining
September 13, 20259 min read
Check the tools your LLM uses before replacing it with just a more powerful model
January 3, 20268 min read
A solution where the "goto" operator becomes irreplaceable.
September 28, 202636 min read
A step-by-step guide to creating your first neural network
August 26, 20228 min read
Machine Learning can be easy and intuitive - here's a complete from-scratch guide to Decision Trees.
April 8, 202110 min read
A 5-Step Process to Maximize Your Chances of Success
January 28, 20219 min read
原文为英文,正文由自动抓取并保留原文。
原文发布于 Towards Data Science(How to Oversee AI Agents at Scale With a Three-Tier Architecture),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。