Ollama 新增语音模型支持

ollama run phonon "Hello from Ollama."
在搭载 M5 Max 的 MacBook Pro 上,Ollama 0.40.11 的未编辑输出。
Apple Silicon 上的性能表现
在 Apple Silicon 上,Ollama 由 Apple 的机器学习框架 MLX 加速,可实现顶尖水平的吞吐。在搭载 M5 Max 的 MacBook Pro 上,Ollama 比第二快的引擎——Gradium 自研运行时——快 35%。
相对实时的加速倍数
在搭载 M5 Max 的 MacBook Pro 上测得,生成 45 秒语音。取 10 次运行的中位数。Gradium 自研运行时 ptts 在 CPU 上使用 8 位权重,在神经引擎上使用 fp16,在 GPU 上使用 f32;Ollama 则以 bf16 运行该模型。
Ollama 的实现同样准确,其词错误率与 Hugging Face Transformers 中的参考实现持平:
数值越低越好。取自 Seed-TTS 英文测试集的 200 个句子,使用 Whisper large-v3 转写。
使用 --voice 选择音色,或在 API 中通过 voice 指定:
ollama run phonon --voice Hadley "来自 Ollama 的问候。"
这些音色来自 Gradium 的音色目录。ollama show phonon 会列出全部音色。
phonon 在 Apache 2.0 许可下发布。40M 模型在 CC BY-NC 4.0 许可下发布,这限制了它们只能用于非商业用途。
将聊天模型的回复通过管道传给 Phonon。文本一到就会被朗读,因此回复会在第一句话之后开始播放:
ollama run gemma4 --hidethinking "用两句话告诉我一个关于猫头鹰的有趣事实。" | ollama run phonon
运行 ollama run phonon 但不提供文本,即可在提示符下逐行输入;或者重定向输出以保存 WAV 文件:
ollama run phonon "把这段内容保存到文件。" > speech.wav
Phonon 兼容 OpenAI 的 /v1/audio/speech 端点,因此使用 OpenAI SDK 的现有代码只需换一个新的 base URL。
curl http://localhost:11434/v1/audio/speech \ -H "Content-Type: application/json" \ -d '{ "model": "phonon", "voice": "Zoey", "input": "Hello from Ollama." }' --output speech.wav
from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") with client.audio.speech.with_streaming_response.create( model="phonon", voice="Zoey", input="Hello from Ollama.", response_format="wav", ) as response: response.stream_to_file("speech.wav")
import fs from "node:fs" import OpenAI from "openai" const openai = new OpenAI({ baseURL: "http://localhost:11434/v1", apiKey: "ollama" }) const response = await openai.audio.speech.create({ model: "phonon", voice: "Zoey", input: "Hello from Ollama.", response_format: "wav", }) fs.writeFileSync("speech.wav", Buffer.from(await response.arrayBuffer()))
响应会在音频生成的同时进行流式传输,因此可以立即播放:
curl -sN http://localhost:11434/v1/audio/speech \ -H "Content-Type: application/json" \ -d '{"model": "phonon", "input": "Playback starts right away."}' \ | ffplay -nodisp -autoexit -loglevel error -
音频为 24 kHz 单声道,默认以 WAV 返回,若设置 "response_format": "pcm" 则返回原始 16 位采样。
语音模型在 Apple Silicon Mac 上运行。下载 Ollama 0.40.11 或更高版本,然后拉取 Phonon:
更多信息,请参见:
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Speech models in Ollama)
ollama run phonon "Hello from Ollama."
Unedited output from Ollama 0.40.11 on a MacBook Pro with M5 Max.
Performance on Apple Silicon
On Apple Silicon, Ollama is accelerated by MLX, Apple’s machine learning framework, for state-of-the-art throughput. On a MacBook Pro with M5 Max, Ollama was 35% faster than the next fastest engine, Gradium’s own runtime.
Times faster than real time
Measured on a MacBook Pro with M5 Max, generating 45 seconds of speech. Median of 10 runs. Gradium's own runtime, ptts, uses 8-bit weights on the CPU, fp16 on the Neural Engine and f32 on the GPU; Ollama runs the model in bf16.
Ollama’s implementation is just as accurate, matching the word error rate of the reference implementation in Hugging Face Transformers:
Lower is better. 200 sentences from the Seed-TTS English test set, transcribed with Whisper large-v3.
Choose a voice with --voice, or voice in the API:
ollama run phonon --voice Hadley "Hello from Ollama."
The voices come from Gradium’s voice catalog. ollama show phonon lists them all.
phonon is released under Apache 2.0. The 40M models are released under CC BY-NC 4.0, which limits them to noncommercial use.
Pipe a chat model’s reply into Phonon. Text is spoken as it arrives, so the reply starts playing after its first sentence:
ollama run gemma4 --hidethinking "Tell me a fun fact about owls in two sentences." | ollama run phonon
Run ollama run phonon with no text to type lines at a prompt, or redirect the output to save a WAV file:
ollama run phonon "Save this to a file." > speech.wav
Phonon works with the OpenAI-compatible /v1/audio/speech endpoint, so existing code that uses the OpenAI SDK only needs a new base URL.
curl http://localhost:11434/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{
"model": "phonon",
"voice": "Zoey",
"input": "Hello from Ollama."
}' --output speech.wav
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
with client.audio.speech.with_streaming_response.create(
model="phonon",
voice="Zoey",
input="Hello from Ollama.",
response_format="wav",
) as response:
response.stream_to_file("speech.wav")
import fs from "node:fs"
import OpenAI from "openai"
const openai = new OpenAI({ baseURL: "http://localhost:11434/v1", apiKey: "ollama" })
const response = await openai.audio.speech.create({
model: "phonon",
voice: "Zoey",
input: "Hello from Ollama.",
response_format: "wav",
})
fs.writeFileSync("speech.wav", Buffer.from(await response.arrayBuffer()))
The response streams while the audio is generated, so it can play right away:
curl -sN http://localhost:11434/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model": "phonon", "input": "Playback starts right away."}' \
| ffplay -nodisp -autoexit -loglevel error -
Audio is 24 kHz mono, returned as WAV by default or as raw 16-bit samples with "response_format": "pcm".
Speech models run on Apple Silicon Macs. Download Ollama 0.40.11 or later, then pull Phonon:
For more information, see:
原文为英文,正文由自动抓取并保留原文。
原文发布于 Ollama Blog(Speech models in Ollama),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。