5 分钟读懂决策 AI

了解决策模型如何运作、它们如何将 LLM 变成更快的决策系统,以及如何在本地运行开源替代方案。
作者:Abid Ali Awan,KDnuggets 助理编辑,2026 年 11 月 20 日,人工智能栏目
并非每项 AI 任务都需要一个生成文本的大语言模型(LLM)。有时,我们只需要一个模型来快速做出决策,比如选择合适的工具、对客户投诉进行分类,或判断某笔交易是否可疑。
这正是 Decision AI 的用武之地。随着 TypeSafe AI 发布 Jev 以及若干开源替代方案,我们现在有了专门为做出这类决策而设计的模型,无需生成多余的文本。
在本文中,我们将了解 Decision AI 的运作方式,探索开源模型,并使用 llama.cpp 在本地运行我们的第一个决策模型。
Decision AI 是指利用 AI 来辅助做出决策。这是一个宽泛的领域,但新一代决策模型聚焦的方向相当简单:模型不从零生成文本,而是从预定义答案中进行选择并返回概率。
这类模型通常支持三类问题:
选择(Choice):从选项中选取一个,例如账单、销售或技术支持。
评分:按某种量表对事物进行评级,例如客户投诉的紧急程度。
是或否:判断某件事是否为真,例如一封电子邮件是否为垃圾邮件。
例如,假设一位客户发来电子邮件说:“我的订单被重复扣费了。”
我们不需要 LLM 来写冗长的解释。决策模型可以直接将计费部门识别为最合适的部门,并为该答案返回一个概率。
这使得决策模型适用于处理成千上万个类似请求的应用。
决策 AI 如何运作?
过程很直接。我们向模型提供一些信息,定义一个问题,并指定可能的答案。模型评估这些信息,并返回带有概率的决策。
以下是一个简单的工作流程:
回到我们的客户支持示例,一个说明性的结果可能如下所示:
它需要人工审核吗?
这些是假设性的结果。在实践中,我们会在进行任何自动化之前,先用真实的客户消息测试模型的准确率和概率。
主要优势在于答案是结构化的,因此我们的应用可以直接使用它们,而无需从生成的段落中提取信息。
将大语言模型转变为决策模型
我们可以通过使用现有 LLM 的内部预测而非生成文本,将一个现有的 LLM 转变为决策模型。
当 LLM 处理一个输入时,它会产生 logits,即各个可能的下一个 Token 的原始分数。我们可以提取我们预定义答案对应的分数,并应用一个名为 softmax 的数学函数,将它们转换为概率。
例如,在对一条产品评论进行分类时,模型可能会产生:
应用 softmax 后,我们得到大约 73% 正面和 27% 负面。这些概率是相对于我们所选的选项而言的。
我们可以更进一步,用一个小型预测头替换通常的文本生成层,并在决策任务上微调模型。这使它能够在单次前向传播中对多个选项进行评分,并有可能回答多个问题。
主要优势在于我们跳过了逐 Token 的文本生成,使重复决策更快、更高效。
开源决策 AI 模型与框架
Jev 是一个专有模型,可通过 TypeSafe 的 API 使用。尽管该公司分享了其方法的一些细节,但尚未公布完整的架构、模型权重或训练配方。因此,我们仍然不知道其底层究竟是如何运作的。
与此同时,开源社区一直在以惊人的速度前进,发布了我们可以检查、微调和本地运行的替代方案。一些方案已经报告在特定基准测试上取得更好的结果、更快的推理速度以及可能更低的运行成本。
以下是五个值得探索的项目:
Laya:一个支持 100 多种语言的轻量级决策模型。它提供快速推理,自动选择合适的模型,并能在普通硬件上运行。
Kev:一系列基于 Qwen 的模型,参数规模从 0.8B 到 27B。它支持长文档、自定义微调,并可通过兼容 Jev 的 API 进行部署。
Simple Jev:让我们无需额外训练,就能直接使用现有的 Hugging Face 模型完成分类与打分。它还支持用视觉模型对图像做出决策。
Decider:专为在单次前向传播中处理多个决策而设计。每个问题最多支持 255 个选项,并包含涉及俄罗斯方块(Tetris)和 Pong 等游戏的实验性演示。
SemIf:聚焦本地决策,支持 NVIDIA GPU、Apple Silicon 和 CPU。它包含一个交互式浏览器演示,以及用于比较准确率和推理速度的可复现基准测试。
有意思的地方在于,我们现在有几种方式可以在不依赖闭源 API 的情况下构建决策类应用。我们可以为了速度选择更小的模型,针对特定任务进行微调,或者把一切放在本地运行以降低 API 成本。
用 llama.cpp 在本地运行决策 AI
事情进展很快。Ollama 和 llama.cpp 现在都通过兼容 TypeSafe 的 /v1/systemone 端点支持决策模型。
Ollama 支持 Nimble 和 Tev1 等模型,而 llama.cpp 支持 Laya、Kev、Julia-1 和 OpenJev。
我们试试用最新的 llama.cpp 来运行 Laya。启动本地服务器:
llama update llama serve -hf ggml-org/Laya-GGUF
模型运行起来后,打开另一个终端,发送一条客户支持请求:
curl http://localhost:8080/v1/systemone \ -H "Content-Type: application/json" \ -d '{ "state": "I was charged twice for my order.", "questions": { "department": { "type": "choice", "instructions": "Which team should handle this?", "criteria": { "billing": "Payments and refunds", "shipping": "Delivery problems", "technical": "Bugs and account issues" } } } }'
模型会评估这条消息,并返回所选部门以及每个选项对应的概率。
就是这样。现在我们无需使用付费 API,就能在本地试用决策模型了。
决策 AI 的未来
我认为决策 AI 最有趣的一点,是它能与现有 LLM 协同工作,让 AI 应用变得更快、更便宜。
设想一个 AI 编程智能体,用一个小型决策模型来选择正确的工具、决定是否需要搜索网络,或判断某项任务是否需要更强大的 LLM。与其为每一个细小的决策都动用昂贵的推理模型,我们可以把它留给真正需要它的任务。
另一个令人兴奋的领域是实时应用。决策模型已经在游戏中进行测试,它们能在毫秒级内选出下一步动作。我们可能会在机器人、欺诈检测和自主 AI 智能体中看到类似的应用。
随着 Ollama 和 llama.cpp 现在支持这些模型,我们也可以在本地运行它们,从而降低 API 成本,并将敏感数据保留在自己的硬件上。
最大的挑战是可靠性。如果一个模型表示自己有 95% 的置信度,我们需要知道这个数字是否可信。随着这些模型进入现实世界的应用,更好的训练、校准和人工审核将变得很重要。
Abid Ali Awan(@1abidaliawan)是一位认证数据科学家,热爱构建机器学习模型。目前,他专注于内容创作,并撰写有关机器学习和数据科学技术的技术博客。Abid 拥有技术管理硕士学位和电信工程学士学位。他的愿景是利用图神经网络为受精神疾病困扰的学生打造一款 AI 产品。
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Decision AI Explained in 5 Minutes)
Learn how decision models work, how they turn LLMs into faster decision-making systems, and how to run open-source alternatives locally.
By Abid Ali Awan, KDnuggets Assistant Editor on November 20, 2026 in Artificial Intelligence

Not every AI task needs a large language model (LLM) that generates text. Sometimes, we just need a model to make a quick decision, such as choosing the right tool, classifying a customer complaint, or deciding whether a transaction looks suspicious.
This is where Decision AI comes in. With the release of Jev by TypeSafe AI and several open-source alternatives, we now have models specifically designed to make these decisions without generating unnecessary text.
In this article, we will learn how Decision AI works, explore open-source models, and run our first decision model locally using llama.cpp.
Decision AI is the use of AI to help make decisions. It is a broad field, but a newer generation of decision models focuses on something quite simple: instead of generating text, the model selects from predefined answers and returns probabilities.
These models generally support three types of questions:
Choice: Select an option, such as billing, sales, or technical support.
Score: Rate something on a scale, such as the urgency of a customer complaint.
Yes or no: Estimate whether something is true, such as whether an email is spam.
For example, imagine a customer sends an email saying, "I was charged twice for my order."
We don't need an LLM to write a lengthy explanation. A decision model can simply identify billing as the most appropriate department and return a probability for that answer.
This makes decision models useful for applications that process thousands of similar requests.
How Does Decision AI Work?
The process is straightforward. We provide the model with some information, define a question, and specify the possible answers. The model evaluates the information and returns its decision with probabilities.
Here is a simple workflow:
Going back to our customer support example, an illustrative result might look like this:
Does it need human review?
These are hypothetical results. In practice, we would test the model's accuracy and probabilities on real customer messages before automating anything.
The main advantage is that the answers are structured, so our application can use them directly without extracting information from generated paragraphs.
Turning Large Language Models Into Decision Models
We can turn an existing LLM into a decision model by using its internal predictions instead of generating text.
When an LLM processes an input, it produces logits, which are raw scores for possible next tokens. We can extract the scores for our predefined answers and apply a mathematical function called softmax to convert them into probabilities.
For example, when classifying a product review, the model might produce:
After applying softmax, we get approximately 73% positive and 27% negative. These probabilities are relative to the options we selected.
We can take this further by replacing the usual text-generation layer with a small prediction head and fine-tuning the model on decision-making tasks. This lets it score multiple options, and potentially answer several questions, in a single forward pass.
The main advantage is that we skip token-by-token text generation, making repeated decisions faster and more efficient.
Open-Source Decision AI Models and Frameworks
Jev is a proprietary model available through TypeSafe's API. Although the company has shared some details about its approach, it hasn't published the full architecture, model weights, or training recipe. So we still don't know exactly how everything works under the hood.
Meanwhile, the open-source community has been moving incredibly fast, releasing alternatives that we can inspect, fine-tune, and run locally. Some are already reporting better results on specific benchmarks, faster inference, and potentially lower running costs.
Here are five projects worth exploring:
Laya: A lightweight decision model supporting over 100 languages. It offers fast inference, automatically selects the appropriate model, and runs on modest hardware.
Kev: A family of Qwen-based models ranging from 0.8B to 27B parameters. It supports long documents, custom fine-tuning, and deployment through a Jev-compatible API.
Simple Jev: Lets us use existing Hugging Face models for classification and scoring without additional training. It also supports vision models for making decisions about images.
Decider: Designed to handle multiple decisions in a single forward pass. It supports up to 255 choices per question and includes experimental demonstrations involving games such as Tetris and Pong.
SemIf: Focuses on local decision-making, with support for NVIDIA GPUs, Apple Silicon, and CPUs. It includes an interactive browser demo and reproducible benchmarks for comparing accuracy and inference speed.
The interesting part is that we now have several ways to build decision-making applications without depending on a closed-source API. We can choose smaller models for speed, fine-tune them for our specific tasks, or run everything locally to reduce API costs.
Running Decision AI Locally With llama.cpp
Things are moving quickly. Both Ollama and llama.cpp now support decision models through the TypeSafe-compatible /v1/systemone endpoint.
Ollama supports models such as Nimble and Tev1, while llama.cpp supports Laya, Kev, Julia-1, and OpenJev.
Let's try Laya using the latest llama.cpp. Start the local server:
llama update
llama serve -hf ggml-org/Laya-GGUF
Once the model is running, open another terminal and send a customer support request:
curl http://localhost:8080/v1/systemone \
-H "Content-Type: application/json" \
-d '{
"state": "I was charged twice for my order.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments and refunds",
"shipping": "Delivery problems",
"technical": "Bugs and account issues"
}
}
}
}'
The model will evaluate the message and return the selected department along with a probability for each option.
That's it. We can now experiment with a decision model locally without using a paid API.
The Future of Decision AI
I think the most interesting part of Decision AI is how it can work alongside existing LLMs to make AI applications faster and cheaper.
Imagine an AI coding agent using a small decision model to select the right tool, decide whether to search the web, or determine if a task needs a more powerful LLM. Instead of using an expensive reasoning model for every little decision, we can reserve it for tasks that actually need it.
Another exciting area is real-time applications. Decision models are already being tested in games, where they can choose the next move in milliseconds. We could see similar applications in robotics, fraud detection, and autonomous AI agents.
With Ollama and llama.cpp now supporting these models, we can also run them locally, reducing API costs and keeping sensitive data on our own hardware.
The biggest challenge is reliability. If a model says it is 95% confident, we need to know whether we can trust that number. Better training, calibration, and human review will be important as these models move into real-world applications.
Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in technology management and a bachelor's degree in telecommunication engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.
原文为英文,正文由自动抓取并保留原文。
原文发布于 KDnuggets(Decision AI Explained in 5 Minutes),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。