AI 公司开发决策模型以避免幻觉

致科技内行
致科技内行
IEEE Spectrum 是 IEEE 的旗舰出版物——IEEE 是全球最大的致力于工程与应用科学的专业组织。我们的文章、视频和信息图表让读者了解技术、工程和科学领域的最新动态。
创建账户,享受更多免费内容和权益
将文章保存以便日后阅读,需要一个 IEEE Spectrum 账户
The Institute 的内容仅对会员开放
下载完整 PDF 期刊为 IEEE 会员专属
下载本电子书为 IEEE 会员专属
Spectrum 数字版的访问权限仅限 IEEE 会员专享
关注主题是 IEEE 会员专属的功能
加入全球最大的工程技术及应用科学专业组织,即可访问 Spectrum 的全部文章、档案、PDF 下载及其他权益。了解更多关于 IEEE 的信息 →
加入全球最大的工程技术及应用科学专业组织,即可获取这本电子书,以及 IEEE Spectrum 的全部文章、档案、PDF 下载和其他权益。了解更多关于 IEEE 的信息 →
访问数千篇文章——完全免费
创建账号即可获取专属内容和功能:保存文章、下载合集、发表评论——全部免费!如需完整访问权限和更多权益,请订阅 Spectrum。
AI 公司开发决策模型以避免幻觉
TypeSafe AI 的模型 Jev 牵头推动一项努力,旨在打造更便宜、更快、更可靠的模型
Matthew S. Smith 是 IEEE Spectrum 的特约编辑,曾任 Digital Trends 的首席评测编辑。
Jev 和其他决策模型是一类通过从用户预先定义的少数选项中进行选择来响应查询的模型,以降低计算成本和幻觉。
图片来源:TypeSafe AI
9 月 15 日,一家名为 TypeSafe AI 的初创公司发布了一款名为 Jev 的新 AI 模型。该公司关于 Jev 的广泛且颠覆性的主张引人注目:该公司声称,Jev——它称之为“System One 模型”——更便宜、更快,而且不会产生幻觉。
TypeSafe AI 的营销显然是对其他 AI 公司及其大型语言模型(LLM)的一次警告;这些模型尽管在数学和编程方面能力出众,却以成本高昂和容易“幻觉”出事实错误的回复而臭名昭著。这家初创公司无疑同时吸引了关注和投资。该公司最近以 75 亿美元估值融资 8.7 亿美元。
但 Jev 成功最令人印象深刻的指标,或许可以说是它已经催生的众多模仿者。Cloudflare、Amazon、OpenAI 和 Microsoft 迅速发布了模仿 Jev 能力的“决策模型”。初创公司、学术研究人员和独立爱好者又发布了数十个。
“我得承认 Jev 的功劳,”独立开发者 Matt Mastracci 说,他训练并发布了多个决策模型,包括 Djev,Cloudflare 曾称 Djev 是其自家决策模型的灵感来源。“它确实把决策模型推到了前沿。在六七周时间里,我们已经看到数十个模型问世。”
Jev 的延迟与计算优势
大多数 AI 研究者会把 Jev 称为决策模型或决策器模型。TypeSafe AI 将 Jev 冠以「系统一模型」的品牌定位,源自心理学家 Daniel Kahneman 在其著作《思考,快与慢》中普及的一种认知理论。在书中,Kahneman 区分了快速、本能的思考(系统一)与更缓慢、深思熟虑的思考(系统二)。
TypeSafe AI 认为,现代 LLM 提供的是后者,但在提供前者方面过于缓慢。该公司声称,Jev 可以提供 70 至 500 毫秒的端到端响应延迟。相比之下,LLM 通常至少需要数秒才能给出响应,而更复杂的任务(例如从零开始编写软件)可能需要数小时。
独立测试支持了 TypeSafe AI 的说法。卡内基梅隆大学信息系统教授 Ramayya Krishnan 与几位同事合著的一篇预印本论文发现,在包括 RewardBench 和 JudgeBench 在内的多项用于衡量决策准确性的基准测试中,Jev 的中位延迟为 150 毫秒。而 OpenAI 当时能力最强的模型 GPT-6 Astra 的中位延迟为 1.89 秒。
更快的回复往往也是更便宜的回复,因为它所需的计算时间更少,这一点也体现在 TypeSafe AI 的定价上。该初创公司每百万输入 Token 收费 0.042 美元,输出 Token 免费。相比之下,对于 GPT-6 Astra,ChatGPT 每百万输入 Token 收费 10 美元、每百万输出 Token 收费 50 美元。即便是专为快速、低成本推理而设计的 GPT-6 变体 GPT-6 Luna,每百万输入 Token 也要 0.10 美元、每百万输出 Token 0.50 美元。
Mastracci 表示,更低的延迟和成本带来了机会,可以处理 LLM 目前无法处理的任务。“你可以想想内容审核,比如审核海量涌来的社交媒体帖子,或者一个极其繁忙的聊天室。”
Cloudflare 的产品组经理 Michelle Chen(该公司已发布自己的决策模型,包括 Clef 和 Claric)也看好决策模型能取得的成果。“你大概可以把 80% 的典型 LLM 任务卸载到 System One 模型上,它会快得多,也便宜得多。”
TypeSafe AI 对 Jev 如何实现这种速度提供的细节很少,但 Jev 的开放权重模仿者提供了更多透明度。
例如,以 Cloudflare 的 Clef 为例,这是一个开放权重的决策模型,可以在 Hugging Face 上免费下载。“使用 LLM 时,当你生成一个输出 Token 时,它会查看所有之前的 Token,并预测下一个输出 Token,”Chen 解释道。这个过程按顺序进行,一个 Token 接一个 Token,直到遇到结束 Token。“我们的决策模型不这样做。它们只做一次前向传播,生成对所有输入的一次读取,然后生成一个输出。这就是它快得多的原因。”
决策模型如何泛化任务
基于 LLM 的决策模型,比如 Clef(它基于阿里巴巴的 Qwen LLM),仍然包含 LLM 的神经元和基于 Transformer 的模型架构。它们仍然输入和输出 Token。
不过,模型中负责把这些 Token 转换成文本的部分已被修改,以约束其输出。“Jev 只能以预先定义好的结构化答案格式来回答问题,”复旦大学教授 Zhongyu Wei 说。她也是一项研究的共同作者,该研究对决策模型进行基准测试,并推出了一个开放权重替代方案 SII-Decision。“我们给它一个问题以及 A、B、C 之类的选项,它必须从这些选项中做出选择,并在给出选择的同时返回概率。”
从某种意义上说,决策模型是对分类器的一次回归。分类器,顾名思义,是一类学习把数据划分到不同类别的模型。它们被用于各种各样的任务,包括垃圾邮件过滤和人脸检测。“这类模型并不新鲜,”Chen 说,“这几乎就像是一次重新包装。”
但相比此前的分类器,决策模型带来了一项关键升级:泛化能力。
Mastracci 解释说,分类器传统上是通过针对特定领域的训练构建的。而决策模型则保留了 LLM 的通用智能,这使它们能适用于许多场景。“你可以直接把它拿来用,把它指向你要解决的问题,它就能工作,”Mastracci 说。
Cloudflare 正在将这一优势付诸实践。Chen 表示,在决策模型出现之前,想要使用分类器模型的团队必须「获取大量数据、组建机器学习团队,并针对任务对模型进行微调」。决策模型消除了这一要求,因此一个决策模型可以用于多个领域。例如:在 Cloudflare 的 GitHub 文档仓库中标记垃圾 pull request、识别垃圾邮件和钓鱼邮件、发现钓鱼域名、检测针对 LLM 的提示注入攻击,以及阅读有关互联网骨干网中断的邮件,以判断这些中断是否会影响 Cloudflare 的服务。
决策模型非常适合对邮件、消息或文档等数据做出快速判断。来源:TypeSafe AI
决策模型真能阻止幻觉吗?
在编程领域,「类型安全」语言的设计宗旨是:在程序运行之前就拒绝那些混合了不兼容数据类型的操作——比如把数字与字符串相加——而不是在之后产生错误结果。决策模型与此类似,它能防止模型生成意料之外的回复。TypeSafe AI 的公司名称把类型安全编程与决策模型之间的联系变成了其营销的一部分,并以此为基础提出了一个惊人的说法:Jev 永远不会产生幻觉。
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(AI Companies Develop Decision Models to Avoid Hallucinations)
FOR THE TECHNOLOGY INSIDER
FOR THE TECHNOLOGY INSIDER
IEEE Spectrum is the flagship publication of the IEEE — the world’s largest professional organization devoted to engineering and applied sciences. Our articles, videos, and infographics inform our readers about developments in technology, engineering, and science.
Enjoy more free content and benefits by creating an account
Saving articles to read later requires an IEEE Spectrum account
The Institute content is only available for members
Downloading full PDF issues is exclusive for IEEE Members
Downloading this e-book is exclusive for IEEE Members
Access to Spectrum 's Digital Edition is exclusive for IEEE Members
Following topics is a feature exclusive for IEEE Members
Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more about IEEE →
Join the world’s largest professional organization devoted to engineering and applied sciences and get access to this e-book plus all of IEEE Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more about IEEE →
Access Thousands of Articles — Completely Free
Create an account and get exclusive content and features: Save articles, download collections, and post comments — all free! For full access and benefits, subscribe to Spectrum .
AI Companies Develop Decision Models to Avoid Hallucinations
TypeSafe AI’s model Jev spearheads an effort to make cheaper, faster, reliable models
Matthew S. Smith is a contributing editor for IEEE Spectrum and the former lead reviews editor at Digital Trends.

Jev and other decision models are models that responds to queries by choosing between a few options pre-defined by the user, to cut down on compute cost and hallucinations.
Source images: TypeSafe AI
On 15 September, a startup called TypeSafe AI released a new AI model called Jev. The company’s broad, disruptive claims about Jev that turned heads: The company claimed that Jev, which it calls a “System One model” is cheaper, faster, and can’t hallucinate.
TypeSafe AI’s marketing is an obvious shot across the bow of other AI companies and their large language models (LLMs), which, despite their capabilities in math and coding, are infamous for their cost and tendency to “hallucinate” factually incorrect replies. And the startup has certainly attracted both attention and investment. The company recently raised $870 million in funding at a valuation of $7.5 billion.
But the most impressive metric of Jev’s success is arguably the many imitators it’s already spawned. Cloudflare, Amazon, OpenAI, and Microsoft quickly released “decision models” that imitate Jev’s capabilities. Start-ups, academic researchers, and independent enthusiasts have released dozens more.
“I have to give Jev credit,” says Matt Mastracci, an independent developer who has trained and released several decision models including Djev, which Cloudflare cited as an inspiration for its own decision models. “It really brought decision models to the forefront. In six or seven weeks, we’ve seen dozens of models come out.”
Jev’s Latency and Compute Advantages
Most AI researchers would call Jev a decision model or decider model. TypeSafe AI’s branding of Jev as a System One model refers to a theory of cognition popularized by psychologist Daniel Kahneman in his book Thinking, Fast and Slow. In the book, Kahneman draws a line between fast, instinctive thought (System One) and slower, deliberate thought (System Two).
TypeSafe AI argues that modern LLMs offer the second, but are too slow to provide the first. The company claims that Jev can provide end-to-end response latency between 70 and 500 milliseconds. By comparison, LLMs typically require at least several seconds, to provide a response, and more complicated tasks (like coding software from scratch) can take several hours.
Independent testing backs up TypeSafe AI’s claims. A pre-print co-authored by Ramayya Krishnan, a professor of information systems at Carnegie Mellon University, along with several colleagues, found that Jev delivered a median latency of 150 milliseconds across several benchmarks designed to gauge decision accuracy, including RewardBench and JudgeBench. OpenAI’s most capable model at the time, GPT-6 Astra, delivered a median latency of 1.89 seconds.
A faster reply is often a cheaper reply, because it requires less compute time, and that’s reflected in TypeSafe AI’s pricing. The startup charges US $0.042 per million input tokens and output tokens are free. By contrast, for GPT-6 Astra, ChatGPT charges $10 per million input tokens and $50 per million output tokens. Even GPT-6 Luna, a variant of GPT-6 designed for fast, inexpensive inference, costs $0.10 per million input tokens and $0.50 per million output tokens.
Mastracci says that lower latency and costs open opportunities to tackle tasks LLMs currently can’t handle.“You can think of content moderation, such as moderating a fire hose of social media posts, or an extremely busy chat room.”
Michelle Chen, a group product manager at Cloudflare (which has released its own decision models, including Clef and Claric), is also bullish on what decision models can accomplish. “You can probably offload 80 percent of your typical LLM tasks to a System One model that is going to be so much faster and so much cheaper.”
TypeSafe AI has offered few details about how Jev achieves its speed, but Jev’s open-weights imitators provide more transparency.
For example, take Cloudflare’s Clef, an open-weights decision model which is free to download on Hugging Face. “With an LLM, when you’re generating an output token, it looks at all the previous tokens, and predicts the next output token,” explains Chen. This occurs in sequence, token after token, until an end token is reached. “Our decision models don’t do that. They just do a single forward pass, to generate one read of all the inputs, and then generate one output. That’s why it’s a lot faster.”
How Decision Models Generalize Tasks
Decision models based on an LLM, such as Clef (which is based on Alibaba’s Qwen LLM), still contain the neurons and transformer-based model architecture of an LLM. They still input and output tokens.
However, the portions of the model that turn these tokens into text have been changed to constrain the output. “Jev can only answer questions with a predefined, structured answer format,” says Zhongyu Wei, a professor at Fudan University who is also a co-author of a study that benchmarks decision models and introduces an open-weight alternative, SII-Decision. “We give it a question and options such as A, B and C, and it must choose from those options, returning probabilities alongside its choice.”
In one sense, decision models are a return to classifiers, a type of model that, as the name implies, learn to sort data into classes. They’re used in a wide range of tasks, including spam filters and face detection. “These kinds of models are not new,” says Chen. “It’s almost like a rebranding.”
But decision models offer a key upgrade over prior classifiers: generalization.
Mastracci explains that classifiers were traditionally created through domain-specific training. Decision models, though, retain the general intelligence of an LLM, and that makes them applicable to many situations. “You can pick it up off the shelf, you point it at your problem, and it works,” says Mastracci.
Cloudflare is putting this advantage into practice. Before decision models, teams looking to use a classifier model had to “get a lot of data, stand up a machine learning team, and fine-tune” a model for the task, Chen says. Decision models eliminate this requirement, so one decision model can be used in many domains. Examples include flagging spam pull requests in Cloudflare’s GitHub documentation repository, catching spam and phishing emails, spotting phishing domains, detecting prompt-injection attacks on LLMs, and reading emails about internet backbone outages to judge whether they’ll affect Cloudflare’s services.
Decision models are well suited for quick judgements on data like emails, messages, or documents.Source: TypeSafe AI
Do Decision Models Really Stop Hallucinations?
In programming, “type-safe” languages are designed so that operations mixing incompatible kinds of data—adding a number to a string, say—are rejected before the program runs rather than producing a bad result later. Decision models are similar in that they prevent the model from generating an unexpected reply. TypeSafe AI’s name makes the connection between type-safe programming and decision models part of the company’s marketing, and it builds on that to make a startling claim: Jev will never hallucinate.
This is, strictly speaking, true. LLMs hallucinate because they calculate probabilities for tokens repeatedly across a huge vocabulary of possibilities and output many tokens. This leaves room for less probable tokens to lead a response astray, or for the model to deliver tokens that are probable based on the model’s training, but not correct in the context of the reply. Decision models, by contrast, only calculate probabilities among the pre-determined possibilities, and the most probable is chosen. This eliminates hallucination.
However, Carnegie Mellon’s Krishnan cautions that “eliminating hallucinations in the output does not eliminate errors in decision-making.” He and his co-authors tested Jev by asking it to determine if answers to questions were correct. Jev performed well when the judgment could be easily reached by reading the prompt, but performance dropped off when the judgment required math or involved a logic puzzle.
Wei and his co-authors report similar results in their own research. They found the best decision models can slightly beat the accuracy of LLMs when asked questions about document understanding, and score close to LLMs when asked questions related to tool use (such as choosing the correct software tool to call to perform a user’s task). However, decision models fall behind when asked questions in domains that require topic expertise, including medicine, finance, and law.
“I think reasoning is a key factor,” says Wei. Reasoning allows an LLM to work though its response before it provides a final answer to the user. However, Wei adds that reasoning alone doesn’t explain the domain gaps in all cases, as some LLMs performed well even without reasoning. Wei says the methods used to train decision models can also be a factor, and used decision-specific training for SII-Decision.
Yet decision models and LLMs don’t have to be competitors. Krishnan and his colleagues also tested passing Jev’s low-confidence answers to GPT-6 for additional consideration. The combination of models slightly beat the accuracy of GPT-6 alone, and did so at 41 percent of the cost.
Wei thinks this “hybrid architecture” will become common. “The decision model can call an LLM to give a more detailed explanation. And an LLM can call a decision model for a quick judgment. I think that’s very flexible.”
Related Articles Around the Web
原文为英文,正文由自动抓取并保留原文。
原文发布于 IEEE Spectrum AI(AI Companies Develop Decision Models to Avoid Hallucinations),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。