Ollama 现支持 Jev 风格决策模型

在本地运行时延迟更低
今日起可通过 Ollama 使用三款新的决策模型
从 Ollama 0.35 起,可通过新的 /v1/systemone 端点调用这一新 API。将文本作为状态发送,并附带一组具名问题,运行在你机器上的模型会在一次请求中全部作答。这非常适合需要快速决策的任务,例如工单分流、模型路由以及内容或安全审核。
Ollama 上的决策模型速度很快,因为请求无需经过网络传输。在下面的吃豆人(Pac-Man)示例中,Nimble 9B 在 M5 Max 上本地运行时的平均每次决策耗时为 91ms。这足以支持快速决策,例如玩游戏或实时处理内容:
Nimble 9B 在 MacBook Pro M5 Max 上运行,以实时速度回放。
有三款新的决策模型可通过 Ollama 运行:
nimble:由 Bespoke Labs 开发的开源 9B 参数决策模型
tev1:来自 Together AI 的实验性 4B 决策模型
tev1:0.8b:来自 Together AI 的实验性 0.8B 决策模型
更多决策模型即将推出,包括由 Ollama 云提供服务的模型。
Ollama 上的决策模型
Bespoke Labs 公开基准测试 · 准确率,越高越好
在 13 个带人工标签的公开数据集上的平均准确率,覆盖 3,880 项决策。Nimble 和 Tev1 在 Ollama 上进行了评估;Jev 1.13 来自 Bespoke Labs 在同一批决策上发布的运行结果。查看 Ollama 评估结果和基准测试套件。
开始使用时,请先下载或升级到最新版本的 Ollama。接下来,下载一个决策模型,例如 nimble:
你可以通过 curl 或 TypeSafe 官方的 Python SDK 发起请求。
curl http://localhost:11434/v1/systemone -d '{ "model": "nimble", "state": { "ticket": "我被重复扣费了。请退还多收的款项。" }, "questions": { "team": { "type": "choice", "instructions": "这个工单应由哪个团队处理?", "criteria": { "billing": "支付与退款", "technical": "缺陷与集成", "other": "以上都不是" } }, "refund": { "type": "noul", "instructions": "客户是否明确要求退款?" }, "urgency": { "type": "score", "instructions": "这个工单有多紧急?", "criteria": ["常规", "尽快", "紧急"] } } }'
{"model": "nimble", "answers": {"team": {"type": "choice", "choice": "billing", "probabilities": {"billing": 0.985, "technical": 0.012, "other": 0.003}, "confidence": 0.922}, "refund": {"type": "noul", "noul": 0.997}, "urgency": {"type": "score", "score": 0.815, "legend": {"0": "Routine", "1": "Soon", "2": "Urgent"}, "probabilities": {"0": 0.378, "1": 0.429, "2": 0.193}, "confidence": 0.046}}, "usage": {"input_tokens": 841, "output_tokens": 4}}
uv add typesafe-sdk # or: pip install typesafe-sdk export TYPESAFE_BASE_URL=http://localhost:11434 export TYPESAFE_API_KEY=ollama export TYPESAFE_DEFAULT_MODEL=nimble
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient ticket = "I was charged twice. Please refund the extra payment." questions = { "team": Choice( instructions="Which team should handle this ticket?", criteria={ "billing": "Payments and refunds", "technical": "Bugs and integrations", "other": "None of the above", }, ), "refund": Noul( instructions="Does the customer explicitly ask for a refund?", ), "urgency": Score( instructions="How urgent is this ticket?", criteria=["Routine", "Soon", "Urgent"], ), } with TypeSafeClient(timeout=120) as client: result = client.system_one( state={"ticket": ticket}, questions=questions, ) print(result.choices["team"].choice) # billing print(result.nouls["refund"].noul) # 0.997 print(result.scores["urgency"].score) # 0.815
这是未来众多版本中的第一个,将为 Ollama 加入决策模型支持。后续更新将包括:
由 MLX 驱动,在 Apple Silicon 上实现更快的性能
更多专注于不同类型决策的模型
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Ollama now supports Jev-style decision models)
Lower latency when run locally
Three new decision models available today via Ollama
This new API is available as of Ollama 0.35 by using the new /v1/systemone endpoint. Send text as state with a set of named questions, and a model running on your machine answers them all in one request. This is great for tasks that require fast decisions, such as ticket triage, model routing, and content or safety moderation.
Decision models on Ollama are fast, as requests don’t have to travel over a network. Nimble 9B averaged 91ms per decision in the Pac-Man example below when running locally on an M5 Max. That’s fast enough to make rapid decisions such as playing a game or processing content in real time:
Nimble 9B running on a MacBook Pro M5 Max, replayed at real-time speed.
Three new decision models are available to run via Ollama:
nimble: open-source 9B parameter decision model developed by Bespoke Labs
tev1: an experimental 4B decision model from Together AI
tev1:0.8b: an experimental 0.8B decision model from Together AI
More decision models are coming soon, including models served by Ollama’s cloud.
Decision models on Ollama
Bespoke Labs public benchmarks · accuracy, higher is better
Mean accuracy across 13 public data sets with human labels, covering 3,880 decisions. Nimble and Tev1 were evaluated on Ollama; Jev 1.13 is from Bespoke Labs' published run on the same decisions. See the Ollama evaluation results and benchmark suite.
To get started, first download or upgrade to the latest version of Ollama. Next, download a decision model such as nimble:
You can make a request via curl or via TypeSafe’s official Python SDK.
curl http://localhost:11434/v1/systemone -d '{
"model": "nimble",
"state": {
"ticket": "I was charged twice. Please refund the extra payment."
},
"questions": {
"team": {
"type": "choice",
"instructions": "Which team should handle this ticket?",
"criteria": {
"billing": "Payments and refunds",
"technical": "Bugs and integrations",
"other": "None of the above"
}
},
"refund": {
"type": "noul",
"instructions": "Does the customer explicitly ask for a refund?"
},
"urgency": {
"type": "score",
"instructions": "How urgent is this ticket?",
"criteria": ["Routine", "Soon", "Urgent"]
}
}
}'
{
"model": "nimble",
"answers": {
"team": {
"type": "choice",
"choice": "billing",
"probabilities": {"billing": 0.985, "technical": 0.012, "other": 0.003},
"confidence": 0.922
},
"refund": {"type": "noul", "noul": 0.997},
"urgency": {
"type": "score",
"score": 0.815,
"legend": {"0": "Routine", "1": "Soon", "2": "Urgent"},
"probabilities": {"0": 0.378, "1": 0.429, "2": 0.193},
"confidence": 0.046
}
},
"usage": {"input_tokens": 841, "output_tokens": 4}
}
uv add typesafe-sdk # or: pip install typesafe-sdk
export TYPESAFE_BASE_URL=http://localhost:11434
export TYPESAFE_API_KEY=ollama
export TYPESAFE_DEFAULT_MODEL=nimble
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
ticket = "I was charged twice. Please refund the extra payment."
questions = {
"team": Choice(
instructions="Which team should handle this ticket?",
criteria={
"billing": "Payments and refunds",
"technical": "Bugs and integrations",
"other": "None of the above",
},
),
"refund": Noul(
instructions="Does the customer explicitly ask for a refund?",
),
"urgency": Score(
instructions="How urgent is this ticket?",
criteria=["Routine", "Soon", "Urgent"],
),
}
with TypeSafeClient(timeout=120) as client:
result = client.system_one(
state={"ticket": ticket},
questions=questions,
)
print(result.choices["team"].choice) # billing
print(result.nouls["refund"].noul) # 0.997
print(result.scores["urgency"].score) # 0.815
This is the first of many releases to come adding decision model support to Ollama. Future updates will include:
Faster performance on Apple Silicon powered by MLX
More models specializing in different kinds of decision making
原文为英文,正文由自动抓取并保留原文。
原文发布于 Ollama Blog(Ollama now supports Jev-style decision models),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。