11-19 更新 实时
首页/ 深度/ 正文

AI 智能体正考验零信任安全极限

深度 来源:IEEE Spectrum AI 2026-11-10 00:25 0 阅读 机器翻译
AI 智能体正考验零信任安全极限

信任受到考验:智能体式 AI 如何考验零信任 - IEEE Spectrum

AI 智能体正在考验零信任安全的极限

面向技术内幕人士

IEEE Spectrum 是 IEEE 的旗舰出版物——IEEE 是全球最大的致力于工程与应用科学的专业组织。我们的文章、视频和信息图向读者介绍技术、工程和科学领域的最新发展。

创建账户,享受更多免费内容和权益

保存文章以便稍后阅读需要 IEEE Spectrum 账户

The Institute 的内容仅对会员开放

下载完整 PDF 期号为 IEEE 会员专享

下载本电子书仅限 IEEE 会员

访问 Spectrum 数字版仅限 IEEE 会员

关注主题是仅限 IEEE 会员的功能

加入全球最大的致力于工程与应用科学的专业组织,即可访问 Spectrum 的全部文章、档案、PDF 下载及其他权益。了解更多关于 IEEE →

加入全球最大的致力于工程与应用科学的专业组织,即可获取本电子书以及 IEEE Spectrum 的全部文章、档案、PDF 下载及其他权益。了解更多关于 IEEE →

访问数千篇文章 — 完全免费

创建账户即可获取专属内容与功能:保存文章、下载合集、发表评论 — 全部免费!如需完整访问权限与全部权益,请订阅 Spectrum。

AI 智能体正在考验零信任安全的极限

当自主 AI 智能体协同工作时,验证它们的行动、执行安全策略并维持问责变得愈发困难。

Vishal Sharma 是英国北爱尔兰贝尔法斯特女王大学电子、电气工程与计算机科学学院的 Reader(准教授)。

在针对飞行器、采用人类与智能体在环方式的去认证攻击中,AI 智能体可以快速扫描活跃信道并自主拦截流量,从而缩短做出反应并重新获得控制所需的时间。

贝尔法斯特女王大学

本文由贝尔法斯特女王大学提供。

全球已有多个团队在利用 AI:他们组建由编排器管理的智能体特遣队,并开发应用来支持风险评估、客户画像、匹配以及信息产出,同时对决策过程进行轻量级监督。这些智能体的重要组成部分之一是确保日志得到管理,且不得使用历史相关性,以避免决策偏向同一结果。因此,系统需要设置检查点,在这些检查点提示智能体删除此类历史相关性。

然而,尽管采用了这样的方法,未来的提示仍可能让智能体重新调出本应删除的信息,并被诱骗泄露更多信息,这就是典型的上下文恢复攻击。因此,这里提出的问题是:我们如何相信自己发出的指令已被遵循?而证据又在哪里?

在编排和管理智能体活动时,需要引入零信任架构(ZTA)的思路,这其实意味着所有智能体交互都遵循“永不信任、始终验证”这一核心原则。ZTA 的组件包括策略执行点(PEP),而该组件本身必须被信任,才能在整个系统中维持信任,因此仍然存在的问题是:零信任的组件究竟在哪里?我们是否应该信任由 PEP 执行的策略?

在具备恰当防护机制的前提下,采取支持智能体的思路并使用智能体可能会有所助益。但如果把我们的系统向那些基于现成模型训练出的随机智能体开放,并盲目依赖它们,就可能带来重大的网络安全风险。例如,通过使用 AI 模型辅助开发软件的 vibe coding,把整个代码库交给组织外部的不可信实体,可能导致漏洞在多个面上扩散。

通过使用 AI 模型辅助开发软件的 vibe coding,把整个代码库交给组织外部的不可信实体,可能导致漏洞在多个面上扩散。

这引出了关键的讨论:vibe coding 可能如何干扰智能体查询的可信执行这一定位与实现。设想一个自主的智能体网络,它依靠各智能体的输入来驱动机器运作。当系统在低容错条件下运行,且采用不基于追踪的智能体工作流时,实时信任并验证每一次查询在算力上可能非常昂贵,从而导致内存与执行之间常见的取舍。

另一个需要考虑的风险是零日攻击。当它发生时,系统会如何反应?这样的事件能否在不导致基础设施全面崩溃的情况下得到管控?从对只有单一信任层的智能体生态系统的过度依赖中,可以看出其影响规模。这可能意味着,一旦发生故障,整个生态系统都可能暴露,就像骨干网络不可用就没有自主性一样,而这实质上意味着没有任何可以寄托信任的层。

解除认证攻击能以多快的速度发生?

贝尔法斯特女王大学(QUB)开展的一项内部实验,让人们初步了解到智能体攻击的规模可能意味着更短的可利用窗口。该团队聚焦于非常常见的解除认证攻击,而在智能体编排下,发起解除认证攻击的速度最快可低至 2 到 3 秒。这几乎不给人留出反应并决定修补系统的时间。

大规模智能体系统还有另一个方面,即在异步自主下建立信任。信任是可追溯性的关键基础。你如何确保所有朝着共同目标运行的智能体保持一致,并理解与其目标相关的延迟?

我们团队在这一领域开展了工作,起初是证明该问题确实存在,以及当必须在异步自主的边界内证明信任时,其影响和规模可能非常巨大。针对此类问题的解决方案可以通过保留 AI 物料清单(AI-bill of material)之类的做法,或利用分布式账本技术来简化——后者能够通过可追溯性和不可篡改性提供底层的信任基础设施,并使智能体活动及相关信任向量与平台无关。

在信任范畴内,还有一个尚未被充分探讨、但需要深入研究的概念,即横向异常(horizontal anomalies)。它描述的是智能体在不同环境下与各类来源交互方式存在差异的现象。当智能体在某一特定任务或应用中行为正确、并符合架构预期时,并不意味着它们在所有相关组件上的运行都值得信任。如何识别这类异常,以及可以采取哪些方法来应对而不必迁就这类智能体,仍是悬而未决的问题和有待进一步探索的课题。

关键在于,处于中低信任治理水平下的智能体,其效用会带来短期的高收益,但系统失败时的恢复成本尚待探究,而且这不会是一个可控的问题。我们正在面对这样一种系统:智能体已经渗透进不同的实体,且可能无法实现完全恢复。例如,仅仅因为某个智能体失控而引发的供应链问题,如今已不再是一项可控的小任务,而是需要对其波及的所有受影响智能体的接触点进行全面排查。这实质上意味着,尽管智能体数量众多、且能够触发未必与更广泛生态直接关联的代理智能体,仍有必要确保为所有智能体部署计算开销高昂的追踪器。

我们研究组开展的其他相关博士研究工作则面向 AI 主权,探索信任与问责的各个层面。我们的假设是,主权与信任需要被协同治理,因为智能体的宿主在溯源方面承担的责任很重。因此,这需要理解规范基于信任的 AI 执行的相关法律。这里的重要挑战在于,遗漏信息与遗漏指令之间存在关键差异,而在可信问责方面,二者带来的影响各不相同。

真正的零信任架构存在吗?

这些系统在现实世界中的问题在于,在治理信任时未能采用那些历史上依赖基于状态的建模的方法。在智能体自主性很高的无状态系统中,向信任—溯源收敛的能力与可靠性会显著下降。因此,这又把我们带回了最初的问题——真正的零信任究竟存在吗?

原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(AI Agents Are Testing the Limits of Zero-Trust Security)

Trust Under Fire: How Agentic AI Tests Zero-Trust - IEEE Spectrum

AI Agents Are Testing the Limits of Zero-Trust Security

FOR THE TECHNOLOGY INSIDER

IEEE Spectrum is the flagship publication of the IEEE — the world’s largest professional organization devoted to engineering and applied sciences. Our articles, videos, and infographics inform our readers about developments in technology, engineering, and science.

Enjoy more free content and benefits by creating an account

Saving articles to read later requires an IEEE Spectrum account

The Institute content is only available for members

Downloading full PDF issues is exclusive for IEEE Members

Downloading this e-book is exclusive for IEEE Members

Access to Spectrum 's Digital Edition is exclusive for IEEE Members

Following topics is a feature exclusive for IEEE Members

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more about IEEE →

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to this e-book plus all of IEEE Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more about IEEE →

Access Thousands of Articles — Completely Free

Create an account and get exclusive content and features: Save articles, download collections, and post comments — all free! For full access and benefits, subscribe to Spectrum .

AI Agents Are Testing the Limits of Zero-Trust Security

As autonomous AI agents work together, verifying their actions, enforcing security policies, and maintaining accountability becomes increasingly difficult

Vishal Sharma is a Reader in the School of Electronics, Electrical Engineering, and Computer Science at Queen's University Belfast, Northern Ireland, United Kingdom.

Collage of indoor drone flight, phone controls, and operator at computer workstation

In a de-authentication attack on an aerial vehicle using a human- and agent-in-the-loop approach, an AI agent can quickly scan the active channel and autonomously intercept traffic, thus reducing the time to react and regain control.

Queen's University Belfast

This article is brought to you by Queen’s University Belfast.

Several teams around the globe are leveraging AI by creating a task force of agents managed by an orchestrator and creating applications to support risk assessment, client profiling, match making, and information yield with a slight overview of decision making. One of the important components for these agents is to ensure that the logs are managed and historic relevance is not being used to avoid skewing towards same decision making. Thus, the system is expected to keep checkpoints in place where agents are prompted to delete such historic relevance.

However, despite such methodology in place, a future prompt could make the agents bring forward the information that was supposed to be deleted and can be tricked into revealing more information, which is a classic case of context recovery attack. So, the question raised here is: How can we trust that our instructions have been followed? And where is the proof?

There is an element of utilizing zero-trust architectures (ZTA) for orchestrating and managing agentic activity, which simply means all agentic interactions use the core principle of “never trust,” “always verify.” ZTA components include a policy enforcement point (PEP) that has to be trusted to maintain trust across the system, so the question that remains is where is the component of zero-trust, and shall we trust the policies enforced by PEP?

Having a pro-agentic approach and utilizing agents could be helpful with the right guardrails in place. Opening our systems to random agents that are being trained on off-the-shelf models and blindly relying on them can pose significant cyber risks. For example, vibe coding by using an AI model to help develop software, giving away the entire codebase to an untrusted entity outside the organization can lead to vulnerability sweeping across multiple surfaces.

Vibe coding by using an AI model to help develop software, giving away the entire codebase to an untrusted entity outside the organization can lead to vulnerability sweeping across multiple surfaces.

This opens critical discussions towards how vibe coding could interfere with the positioning and implementation of trusted execution of the agentic queries. Consider an autonomous web of agents that relies on agents’ inputs to drive the operations. When the system is operating under low tolerance of failure and with non-tracking based agentic workflows, it could be computationally expensive to trust and verify each query in real-time that leads to the common trade-offs of memory vs execution.

Another risk to consider is the zero-day attack. When it happens, how will the system react? Would such an event be manageable without a total infrastructure breakdown? The scale of impact can be observed from over reliance on agentic ecosystems with only one trusted layer. This could mean that when failure occurs, the entire ecosystem could be exposed, just like there will be no autonomy if the backbone network is not available, which essentially means no layer to place trust.

How quickly can de-authentication attacks occur?

An in-house experiment done at Queen’s University Belfast (QUB) provided an initial understanding of how the scale of agentic attacks could mean a lower window of vulnerability. The team focused on very popular de-authentication attacks and with agentic orchestration, the swift rate to do de-authentication attacks could go as low as 2 to 3 seconds. This gives little to no time to react and decide on patching the system.

There is another aspect of the large-scale agentic system, which establishes trust under asynchronous autonomy. Trust serves as a critical foundation for traceability. How would you ensure that all the agents operating towards a common goal are aligned and understand the delays associated with their goals?

Our team worked around this space, which at first involved demonstrating that the problem exists, and its impact and scale can be huge when you must prove trust under boundaries of asynchronous autonomy. Solutions towards such problems can be simplified by keeping things like AI-bill of material or leveraging distributed ledger technology that can provide underlying infrastructure of trust via traceability and immutability and keep the agentic activity and associated trust vectors agnostic to the platform.

Within trust, another factor that is not well explored but needs a thorough investigation is the concept of horizontal anomalies. It defines how agents interact differently with various sources under different settings. As agents behave correctly and align themselves with the architectural expectations for one specific task or application, that does not necessarily mean these can be trusted for their operations across all associated components. How to identify such anomalies and what methods can be employed without complying with such agents remain open questions and topics for further exploration.

The critical aspect is that utility of agents with semi to low trust governance offer short-term high benefits, but the cost of recovery when the system fails is yet to be explored, and this will not be a contained issue. We are looking into the system where agents are already sweeping into different entities and may not allow complete recovery. For example, a mere supply chain issue because an agent going rogue is now not a containable small task but needs sweeping across all the touch points of those affected agents. This essentially means despite the number of agents and their ability to trigger proxy-agents, which may not be directly linked to the wider ecosystem, there is a need to ensure that computationally expensive trackers are in place for all the agents.

Other relevant PhD works carried out in our research group are towards sovereignty of AI, exploring aspects of trust and accountability. Our hypothesis is that sovereignty and trust need to be governed together as onus of provenance is high on the host of agents. Thus, it needs understanding of laws governing trust-based executions of AI. The important challenge here is that there is a vital difference between missed information and missed instructions, and both have different impacts when it comes to trusted accountability.

Does true zero-trust architecture exist?

The real-world issue with these systems lies in the failure to adopt approaches that historically have relied on state-based modeling when governing trust. In stateless systems where agentic autonomy is high, the convergence and reliability towards trust-provenance decreases significantly. Thus, it brings us back to the original question – does a true zero-trust even exist?

Can policies be trusted? If so, how should these be managed without falling into the trap of policy-based accessibility and control issues? Despite the argument that PEP can help manage access between the agents, placing a single entity and expecting it to scale towards all agentic interactions is underestimating the problem space.

Formal verification of trust is another part of a trust-based agentic ecosystem. When dealing with AI models, understanding the robustness helps to put trust in place for these models.

To build our case and settle around the challenges of trust embeddings in Agentic AI, within Cyber AI hub, we are exploring opportunities to help companies scale up as well as build their pipeline of agentic applications for commercial viability. However, the element that governs our regular work and decision making are scale of problem and number of agentic deployments, anomaly detection across the agentic workflows, continuous evaluation of the attack surfaces and vulnerability assessment, exploring differences in the pure agentic AI and task-driven multi-agent system.

We are working with industries where trust is a pre-requisite, and these organizations put trust as a guardrail managed via internal policies with a pure focus on securing the systems through policies rather than trusting the third-party agentic service providers. Additionally, trust in services is presented by ensuring constraints around their service models where agentic activities are tracked and managed via logs including management of their token revocation list to avoid replay attacks depending on the type of application. Formal verification of trust is another part of a trust-based agentic ecosystem. When dealing with AI models, understanding the robustness helps to put trust in place for these models. Things like balancing the adversarial accuracy along with certified adversarial robustness can help to determine how much the model can be trusted.

But the question is, can we settle for anything less than foolproof when it comes to deployment of agents in the real world? And if so, how much impact can be controlled, and who is responsible for the remaining component?

原文为英文,正文由自动抓取并保留原文。

本文由采集器自动同步

原文发布于 IEEE Spectrum AI(AI Agents Are Testing the Limits of Zero-Trust Security),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗