11-19 更新 实时
首页/ AI 大模型/ 正文

Lazarus:重编程休眠神经元给 AI 智能体植入后门

AI 大模型 来源:Microsoft Research 2026-11-11 04:52 0 阅读 机器翻译
Lazarus:重编程休眠神经元给 AI 智能体植入后门

微软研究人员发现了“Lazarus”,这是一种后门技术,能够重新编程 AI 模型中的休眠神经元,使原本正常的 AI 智能体在隐藏触发器出现时执行恶意命令。

对于具备工具或终端访问权限的 AI 智能体而言,这一威胁尤为严重。被触发的模型可能在通过标准安全评估的同时,悄悄下载并运行攻击者的代码、窃取机密或执行破坏性命令。

这些后门无需重新训练模型即可植入,且难以通过常规测试检测出来,这使得模型来源溯源和权重完整性成为关键的安全问题。

为了保护客户,Microsoft Foundry 会验证模型来源,将模型与受信任的发布方绑定,并在引入开放权重模型之前扫描其周边软件环境。

微软还在开发更多防御手段,包括由 AI 驱动的权重扫描、用于检测模型异常改动的红队测试工具,以及具备防篡改溯源能力和审计追踪的密码学模型签名。

在微软,研究人员与安全团队并肩协作,研究模型和智能体面临的新兴威胁。随后我们将研究成果转化为产品防御能力和指导建议,持续保护客户。

从更广泛的层面看,Lazarus 是一种应当成为惯例的做法的例证:AI 智能体暴露出新的安全问题,随后由人类研究人员将其系统化、加以研究并予以缓解。

以下是首位发现该威胁的研究人员所作的简要总结。

我们发现,对 LLM 中不到 10 个休眠神经元进行权重重编程,就足以为基于它构建的智能体植入后门,而且无需任何训练。模型能通过标准评估,表现一切正常,直到触发条件出现。触发器可能平淡无奇,比如一次日期变化——例如当时间跨过午夜进入 2027 年,智能体就会在执行一项普通任务的过程中,发出下载并运行攻击者代码的命令。随后,智能体会以你授予它的任何权限执行该命令。

好消息是,这类改动是可以被发现的:只要你让前沿智能体去查看,它们就能识别出异常的权重修改。但没有任何扫描能够毫无疑问地证明一个模型是安全的。可靠的检测需要与原始的、未经修改的权重做 diff 比对,因此溯源非常重要:要清楚模型来自哪里,以及它在开发者与部署环节之间没有被改动过。

被植入后门的模型并不是一类新的威胁。它可以被视为遭到入侵的软件,而相应的防御手段也是安全人士早已熟知的,Satya 最近也撰文谈过:把模型视为潜在内部风险,不要无条件信任它们,限制它们能做什么,并监控它们做了什么。

Microsoft Foundry 为 Azure Direct Models 和 Hugging Face 模型提供关键的安全系统,以确保模型来源的安全可信。这意味着,对于来自可信实验室的模型,Foundry 客户能够免受本文所述那类攻击的威胁。此类攻击需要篡改模型的权重,而 Foundry 通过来源管控引入开放权重模型,将每个模型与其原始发布方绑定,而每个模型卡都会标明发布方。随后,模型在 Microsoft 自有容器内运行,这些容器会单独进行关键漏洞扫描。这些管控措施覆盖了模型的来源以及其周边的软件。

剩余的风险在于,可信供应商自身遭到攻破,例如通过其训练基础设施或内部人员。为在这种情况下加强防护,我们已经着手落实本研究的建议。我们正在与内部 AI 安全团队合作,将其检测技术转化为红队测试框架,并探索在引入开放权重模型时,如何利用前沿智能体进行权重扫描,使其成为一项额外的安全信号。此外还计划让模型带有可验证的来源签名和防篡改审计。

从 Microsoft Threat Intelligence 获取防护与缓解指导。(在新标签页中打开)

以下是我们建议的起点:

这项研究始于 3 月的一次快速实验:我让搭载 Opus 4.5 的 Claude Code 在不进行训练的情况下,硬接几个激活值来给模型植入后门。它做到了。那次实验后来演变成对这套我们如今称为 Lazarus 的方法长达数月的研究。以下是我们学到的内容:

这是 Lazarus 工作原理的简化示意图。

《Lazare, veni foras:重新编程休眠神经元,为 AI 智能体植入后门》一文最初发表于 Microsoft Research。

译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents)
MSR HERO Laz

Microsoft researchers have discovered “Lazarus,” a backdoor technique that can reprogram dormant neurons in an AI model, causing an otherwise normal AI agent to execute malicious commands when a hidden trigger appears.

The threat is especially serious for AI agents with tool or terminal access. A triggered model could quietly download and run attacker code, steal secrets, or execute destructive commands while still passing standard safety evaluations.

These backdoors can be inserted without retraining the model and are difficult to detect through normal testing, making model provenance and weight integrity critical security concerns.

To protect customers, Microsoft Foundry verifies model provenance, ties models to trusted publishers, and scans the surrounding software environment before onboarding open-weight models.

Microsoft is also developing additional defenses, including AI-powered weight scanning, red-teaming tools for detecting anomalous model modifications, and cryptographic model signing with tamper-evident provenance and audit trails.

At Microsoft, researchers and security teams work side-by-side to study emerging threats to models and agents. Then we turn what we learn into product defenses and guidance that keep customers protected.

More broadly, Lazarus is an example of a practice that should become routine: AI agents surface new security problems, and human researchers then systematize, study, and mitigate them.

Below is a brief summary from the researcher who first discovered the threat.

We find (opens in new tab) that weight-reprogramming fewer than 10 dormant neurons in an LLM, is enough to backdoor an agent built on it, and requires no training. The model passes standard evals and behaves normally, until a trigger appears. The trigger can be as mundane as a date change, e.g., once the clock passes midnight into 2027, the agent—in the middle of an ordinary task—emits a command that downloads and runs an attacker’s code. The agent then executes it with whatever access you gave it.

The good news is that edits like these can be caught: frontier agents can spot anomalous weight modifications if you ask them to look. But no scan can prove beyond any doubt that a model is safe. Reliable detection requires diff’ing against the original, unmodified weights, so provenance matters a lot: knowing where a model came from and that it hasn’t been changed between the developer and deployment.

Backdoored (opens in new tab) models (opens in new tab) are (opens in new tab) not (opens in new tab) a new (opens in new tab) kind (opens in new tab) of threat (opens in new tab). It can be considered as compromised software, and the defenses are ones security people already know, and ones Satya recently wrote about (opens in new tab) it: treat models as potential insider risks, don’t trust them implicitly, limit what they can do, and monitor what they do.

What Microsoft does today.

Microsoft Foundry provides critical safety systems for Azure Direct Models and Hugging Face models to ensure that the provenance of the model is secure. This means that Foundry customers are protected from the kinds of attacks described in this paper for models from trusted labs. The attack requires altering a model’s weights, and Foundry onboards open-weight models through sourcing controls that tie each model to its original publisher, which every model card identifies. Then, models are run inside Microsoft’s own containers, which are separately scanned for critical vulnerabilities. These controls cover where a model came from and the software around it.

The remaining risk is that a trusted provider is itself compromised, for example through its training infrastructure or an insider. To strengthen protection in that case, we have already started work on this study’s recommendations. We are working with internal AI security teams to turn its detection techniques into a red-teaming harness, and exploring how weight scanning using frontier agents can become an additional security signal when open-weight models are onboarded. Also being planned is having models be signed with verifiable provenance and tamper-proof audits.

Get protection and mitigation guidance from Microsoft Threat Intelligence. (opens in new tab) 

Here is what we suggest as a starting point:

Infographic titled “Recommendations” presenting two columns of guidance for mitigating risks from backdoored AI models. The left column, “For model hubs and vendors,” lists five recommendations: (1) audit model weights with frontier agents, (2) compare model weights against a declared base model, (3) conduct red-team testing for weight-level attacks, (4) show model provenance and audit status, and (5) disclose risks of third-party checkpoints. The right column, “For enterprises and users,” lists five recommendations: (1) know where models come from, (2) limit agent permissions, (3) monitor agent actions, (4) review consequential actions before execution, and (5) treat audits as evidence rather than guarantees. Recommendations are displayed in numbered boxes, with the fifth item in each column highlighted in a light red background.
(This is not exhaustive; no audit can guarantee safe behavior. These should be one layer of a defense-in-depth approach.)

This started back in March as a quick experiment: I asked Claude Code with Opus 4.5 to backdoor a model by hardwiring a few activations without training. And it did. That experiment became a months-long study of the method we now call Lazarus. Here is what we learned:

  • Agents offer a large attack surface. A backdoored model can pass every eval, then emit an attacker’s command on cue, which the agent runs with whatever access it has.
  • Dormant neurons can be turned into trigger detectors. Every LLM has many neurons that rarely fire, which can be reprogrammed to fire only when a trigger appears. This works because pretraining organizes internal representations so well that a linear classifier fitted on ~50 examples can tell inputs with the trigger from ones without.
  • One reprogrammed neuron can inject one token. Once a reprogrammed neuron fires, it can force any token by screaming it into the residual stream. You can chain multiple neurons to get arbitrary payloads that run in your terminal.
  • It needs inference only compute. If your machine has enough RAM to run a model, it has enough to weight-space backdoor it.
  • This is not a case against open weights. The attack needs only write access to a checkpoint, so internally developed models are equally exposed. A compromised account, pipeline, or storage can be sufficient. What matters is weight integrity and provenance.
  • Weights should be audited like code. Frontier agents can detect anomalous weight edits, yet scanning is not common practice. That should change.
  • [!!!] Claude Code with Opus 4.5 discovered most of this autonomously

This is a simplified depiction of how Lazarus works.

The post Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents appeared first on Microsoft Research.

正文保留英文原文,标题与摘要由机器翻译。

本文由采集器自动同步

原文发布于 Microsoft Research(Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗