AI 文本水印或难满足欧盟新规

欧盟的 AI 水印规则对 OpenAI 的 TextGrain 构成挑战 - IEEE Spectrum
IEEE Spectrum 11 月刊已上线!
AI 文本水印或许无法满足欧盟新规
面向技术内幕人士
IEEE Spectrum 是 IEEE 的旗舰出版物,而 IEEE 是全球最大的致力于工程与应用科学的专业组织。我们的文章、视频和信息图表为读者呈现技术、工程与科学领域的最新进展。
创建账户,即可享受更多免费内容和权益
保存文章以便稍后阅读,需要拥有 IEEE Spectrum 账户
The Institute 的内容仅对会员开放
下载完整 PDF 期刊为 IEEE 会员专属
下载本电子书为 IEEE 会员专属
访问 Spectrum 数字版为 IEEE 会员专属
关注话题是 IEEE 会员专属功能
加入全球最大的致力于工程与应用科学的专业组织,即可访问 Spectrum 的所有文章、档案、PDF 下载及其他权益。了解 IEEE 更多信息 →
加入全球最大的致力于工程与应用科学的专业组织,即可获得本电子书以及 IEEE Spectrum 的所有文章、档案、PDF 下载及其他权益。了解 IEEE 更多信息 →
访问数千篇文章——完全免费
创建账户即可获得专属内容与功能:保存文章、下载合集并发表评论——全部免费!如需完整访问权限与权益,请订阅 Spectrum。
AI 文本水印可能无法满足欧盟新规
OpenAI 的 textGrain 等水印在短文本和经过编辑的文本上表现不佳
Anna Desmarais 是常驻法国的自由记者,报道人工智能、新兴技术、航天与虚假信息。
欧盟现在要求 AI 生成的文本必须带有水印,以便能被识别为 AI 生成。但对 OpenAI 于 10 月发布、为遵守欧盟新法而开发的 textGrain 水印的测试显示,这些水印往往无法在文本被进一步编辑后留存,或者在文本不足 150 词时失效。由于水印应达到何种效果尚无明确的法律基准,textGrain 能否满足欧洲的透明度规则仍不确定;这可能让该公司付出高昂代价,也会随着模型日益精密而令 AI 生成文本更难被检测。
AI 模型会根据已写出的文本,计算接下来各个可能“Token”(即它将要生成的下一个词或词片段)的概率,然后选定一个并重复这一过程,直到回复完成。例如,textGrain 技术报告合著者、宾夕法尼亚大学博士后研究员 Xiang Li 表示,在提示词“today the weather is…”之后,模型可能会在“warm”或“cool”之间权衡,作为可能的下一个词。
水印使用一个密钥生成伪随机值——这些数字由数学过程产生,会影响模型建议的 Token。Li 表示,这会在文本中形成一种统计模式,持有同一密钥的检测器可以据此进行查找。
textGrain 与其他水印的不同之处在于,它把熵——衡量模型选词有多不可预测的一项指标——当作一种资源,在嵌入水印时可以按预算加以分配。开发者可以指定他们希望允许多少熵或不可预测性,textGrain 则会向这一目标调整。
Li 表示,这让开发者能更好地控制“嵌入可检测信号”与“保留模型输出多样性”之间的平衡。“我们可以把熵看成一种资源,一种钱……在我们的方法里,我们能准确确定自己想要消耗多少这种资源。”
AI 水印检测的局限
水印对读者来说不可见。但检测结果呈阳性,并不能说明某个人在一段文字中贡献了多少;而结果呈阴性,也不能证明一段文本就是人写的。
OpenAI 的假阳性率为 1%,也就是说,该检测器被设计成会把大约每 100 段未加水印的文本中的 1 段错误标记。在针对 ELI5 数据库(该数据库汇集了 r/explainlikeimfive 子版块的问题与回答)中心理学问题的测试里,该模型在约 80% 的 200 Token 文本(约 150 词)中检测出了水印,而在 400 Token 文本(约 300 词)中这一比例为 95%。
Li 称,这是文本水印技术的“基本局限”,解决起来会很复杂。他补充说,即使额外增加计算量,短文本也可能不包含足够的水印证据以供可靠检测,而较长的文本通常能提供更多证据。
编辑操作同样会削弱信号。据 OpenAI 称,在对 400 Token 段落进行的评估中,仅将 10% 的词语替换为同义词,检测率就从 92% 降至 66%;替换比例最高达 25% 时,检测率降至 17%。Li 表示,由于水印依赖前面的 Token,改动一个词就可能干扰多个位置的信号。
OpenAI 表示,仅靠水印不足以判定内容是否由 AI 生成。
TextGrain 将“补充”其他来源溯源工具,例如元数据中的内容凭证(content credentials),以及 Google DeepMind 为图像和音频输出开发的 SynthID 水印。OpenAI 未回应置评请求。
欧盟《人工智能法案》的透明度义务
《欧盟人工智能法案》第 50 条规定的透明度义务已于 8 月 2 日生效。该法律要求生成合成音频、图像、视频或文本的系统的提供方,以机器可读的格式标注输出内容,使其可被检测为人工生成或被操纵的内容。根据该法律,技术解决方案必须“在技术可行的范围内做到有效、可互操作、稳健且可靠”。
欧盟的《行为准则》为提供方提供了一种自愿方式,用以说明其将如何合规。该准则不要求对短于 200 Token 的输出添加水印。
巴黎高等研究院(Institute of Advanced Education)AI 伦理方向博士后研究员 Thomas Souverain 表示,这项立法存在多种解读空间。
Souverain 表示,textGrain 背后的早期方法看起来是可靠的,在识别 AI 生成文本方面可能比竞争对手表现更好。OpenAI 称,在其自身的测试中,textGrain 的表现与 Google 的 SynthID Text 相当或更优,但这些结果尚未经过独立验证。在独立研究人员能够对其进行测试之前,并且鉴于 AI Act 的标准仍缺乏可衡量的阈值,现在判断 textGrain 是否符合法律要求还为时过早。
“我们只有一份技术研究……还没有经过同行评审的学术论文,我们[目前]也无法获得[针对 textGrain 的]精确协议,”Souverain 说。
OpenAI 正在与一个有限的研究者群体合作,其中包括康奈尔大学计算机科学助理教授 John Thickstun,以及位于斯洛伐克布拉迪斯拉发的 Kempelen 智能技术研究所(KInIT)的研究人员。
Thickstun 和 KInIT 高级研究员 Dominik Macko 表示,他们刚刚获得访问权限,尚未开始评估该系统。Macko 说,他的团队计划在不同学科领域、语言和文本长度上测试 textGrain 的“稳健性”。
Souverain 表示,如果最终发现 textGrain 不符合欧盟的要求,OpenAI 可能面临最高达其年收入 7% 的罚款。
Souverain 还呼吁 OpenAI 将 textGrain 开源,以便其团队之外的研究人员可以通过尝试攻破该模型来“找出所有的缺陷、改写以及可能的翻译错误”。
“我理解企业的考量,如果检测到重大缺陷、重大弱点,[希望]它在后台被发现,”他说。“但我希望看到更开放的阶段,就像如今的 SynthID 那样,[我们]能够获取其方法。”
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(AI Text Watermarking May Not Be Able to Meet the EU’s New Rules)
EU's AI Watermark Rules Challenge OpenAI's TextGrain - IEEE Spectrum
The November issue of IEEE Spectrum is here!
AI Text Watermarking May Not Be Able to Meet the E.U.’s New Rules
FOR THE TECHNOLOGY INSIDER
IEEE Spectrum is the flagship publication of the IEEE — the world’s largest professional organization devoted to engineering and applied sciences. Our articles, videos, and infographics inform our readers about developments in technology, engineering, and science.
Enjoy more free content and benefits by creating an account
Saving articles to read later requires an IEEE Spectrum account
The Institute content is only available for members
Downloading full PDF issues is exclusive for IEEE Members
Downloading this e-book is exclusive for IEEE Members
Access to Spectrum 's Digital Edition is exclusive for IEEE Members
Following topics is a feature exclusive for IEEE Members
Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more about IEEE →
Join the world’s largest professional organization devoted to engineering and applied sciences and get access to this e-book plus all of IEEE Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more about IEEE →
Access Thousands of Articles — Completely Free
Create an account and get exclusive content and features: Save articles, download collections, and post comments — all free! For full access and benefits, subscribe to Spectrum .
AI Text Watermarking May Not Be Able to Meet the E.U.’s New Rules
Watermarks such as OpenAI’s textGrain struggle with short passages and edited text
Anna Desmarais is freelance journalist based in France, covering artificial intelligence, emerging technologies, space, and disinformation.

The European Union now requires AI-generated text to be watermarked so that it can be identified as such. But tests on OpenAI’s new textGrain watermark, released in October and developed to comply with the new E.U. law, show that these watermarks often do not survive additional editing of the text or when it’s under 150 words. With no clear legal benchmark for how well watermarks should work, it’s uncertain whether textGrain will meet Europe’s transparency rules, which could cost the company significantly and make it difficult to detect AI-generated text as models grow more sophisticated.
AI models calculate the probabilities of different possible next “tokens” (the next word or word fragment it generates) based on the text already written, then select one and repeat that process until the response is complete. After the prompt “today the weather is…,” for example, says Xiang Li, coauthor of the textGrain technical report and a postdoctoral researcher at the University of Pennsylvania, the model might weigh “warm” or “cool” as possible next words.
A watermark uses a secret key to generate pseudorandom values—numbers produced by a mathematical process that influence which token the model suggests. This creates a statistical pattern in the text that a detector with the same secret key can look for, Li says.
What makes textGrain different from other watermarks is that it uses entropy, a measure of how unpredictable a model’s word choices are, as a resource that can be budgeted when embedding a watermark. Developers specify how much entropy or unpredictability they want to allow, and textGrain adjusts to that target.
This gives developers more control over the balance between embedding a detectable signal and preserving variation in the model’s output, Li says. “We can think of entropy as a resource, a kind of money.… In our approach, we can accurately determine how much of that resource we want to consume.”
Limits of AI Watermark Detection
The watermark is invisible to readers. But a positive result does not show how much a person contributed to a passage, and a negative result does not prove that a text was human-written.
OpenAI’s false-positive rate is 1 percent, meaning the detector is designed to wrongly flag about one in 100 unwatermarked passages. In tests on psychology questions from the ELI5 database, which gathers questions and answers from the r/explainlikeimfive subreddit, the model detected watermarks in about 80 percent of 200-token text (approximately 150 words) compared to 95 percent of 400-token text (approximately 300 words).
Li says this is a “basic limitation” of text-watermarking technologies and would be complicated to solve. Short passages may not contain enough watermark evidence for reliable detection, even with additional computation, he adds, while longer passages generally provide more.
Editing also weakens the signal. In an evaluation of 400-token passages, replacing just 10 percent of words with synonyms reduced detection from 92 percent to 66 percent; replacing up to 25 percent cut the detection rate to 17 percent, according to OpenAI. Because the watermark depends on preceding tokens, changing one word can disrupt the signal at several positions, Li says.
OpenAI says watermarking alone is not enough to determine whether content is AI-generated.
TextGrain will “complement” other provenance tools, such as content credentials in the metadata and SynthID watermarks developed by Google DeepMind for images and audio outputs. OpenAI did not respond to a request for comment.
E.U. AI Act Transparency Obligations
The transparency obligations in Article 50 of the E.U. AI Act went into effect on 2 August. The law requires providers of systems generating synthetic audio, images, video, or text to mark outputs in a machine-readable format so they can be detected as artificially generated or manipulated. Technical solutions must be “effective, interoperable, robust and reliable as far as this is technically feasible,” according to the law.
The E.U.’s Code of Practice offers a voluntary way for providers to show how they will be compliant. This code does not require watermarks on outputs shorter than 200 tokens.
Thomas Souverain, a postdoctoral fellow on AI Ethics at the Institute of Advanced Education, in Paris, says the legislation is open to interpretation.
Souverain says that the early methodology behind textGrain seems reliable and could work better than its competitors at identifying AI-generated text. OpenAI says textGrain matched or outperformed Google’s SynthID Text in its own tests, but those results have not been independently verified. Until independent researchers can test it, and while the AI Act’s criteria still lack measurable thresholds, it is too early to say whether textGrain meets legal requirements.
“We only have a technical study…there hasn’t been a peer-reviewed academic article, and we don’t [yet] have access to a precise protocol [for textGrain],” Souverain says.
OpenAI is working with a limited group of researchers that includes John Thickstun, an assistant professor of computer science at Cornell University, and researchers at the Kempelen Institute of Intelligent Technologies (KInIT) in Bratislava, Slovakia.
Thickstun and Dominik Macko, a senior researcher at KInIT, says they had only just gained access and had not yet begun evaluating the system. Macko says his team plans to test textGrain’s “robustness” across different subject areas, languages, and text lengths.
If textGrain is eventually found to not meet the E.U.’s requirements, Souverain says, OpenAI could face a fine of up to 7 percent of its annual income.
Souverain also calls on OpenAI to make textGrain open-source so that researchers not on their team can “identify all the faults, paraphrases and possible translation faults” of the model by trying to break it.
“I see the corporate interest, if there is a big flaw detected, a big weakness, [that] it is detected in the background,” he said. “But I would like to see an even more open phase, such as today with SynthID, [where] we have access to the methods.”
Related Articles Around the Web
原文为英文,正文由自动抓取并保留原文。
原文发布于 IEEE Spectrum AI(AI Text Watermarking May Not Be Able to Meet the EU’s New Rules),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。