越狱 PS4 跑 90 亿参数模型,达 43 Token/秒

开发者 Leonardo Alegría 的开源项目 Garlic LLM 让一台越狱的 PS4 能够在该主机 2013 年时代的 GPU 上离线运行 90 亿参数的 Qwen3.5-9B AI 模型。该应用直接与 GPU 通信,模型被压缩到约 4.5GiB 以适配主机内存,运行速度为每秒 31 至 39 个 Token。
该项目以 GPU 直接读取的内存命名——其同系列内存类型是“Onion”——依赖一台运行 GoldHEN 的 PS4,GoldHEN 是针对已被破解主机的主机自制软件启用器。Garlic LLM 是一个免费的单一 5.66GB 下载包,可在 Hugging Face 上获取,并通过 PS4 的 Package Installer 安装,需要约 6GB 可用空间;若先将安装包复制到主机内置硬盘,则需要 12GB。开发者在固件 11.52 的基础版/Slim PS4 上进行了测试。
它支持语音对话,通过 Chatterbox 语音模型以七种语言进行语音回复,聊天模型运行速度约为每秒 22 个 Token。按住手柄上的一个按键还能让它借助 NVIDIA 的 Parakeet TDT 0.6B v3 语音转文本模型进行听写,该模型支持 25 种欧洲语言。Parakeet 系列模型也为 Meetily、Hex 等本地转写与听写应用提供支持。主模型是阿里巴巴的小型开放权重模型 Qwen3.5-9B;据称其在文本聊天中可达每秒 31 至 39 个 Token——提示词与测试方法未给出——在该主机上可用。
最接近的参照是 PS5LM 这个分支:它在 PS5 上以原生应用的形式运行 llama.cpp,并通过 PS5_Vulkan(RADV 驱动的移植版)访问主机的 AMD GPU。开源的 llama.cpp 是一款被广泛使用的、用于在本地运行语言模型的程序,而 RADV 是让 AMD GPU 能够运行 Vulkan 这一图形与计算 API 的开源驱动。在 PS4 上,自制软件并没有这样的驱动层;Garlic LLM 既没用 llama.cpp,也没用 Vulkan 移植版——至少看起来是这样。
该模型的各种运算,比如对 4 bit 权重做矩阵乘法,都是用 OpenCL C 编写的计算内核,并用 LLVM 为 AMD 的 Graphics Core Next(GCN)架构编译。权重保存在那块“Garlic”内存中,而 CPU 一侧则用纯 C 处理分词器、界面、文本渲染和音频。整条流水线从提示词开始,到 CPU 分词器,随后针对每个 Token 生成一个 PM4 命令缓冲区——这本该由图形驱动生成的底层命令,在这里由应用自己构建——再交给 GCN 内核,最后 Token 被显示或朗读出来。整个项目是用开源 OpenOrbis PS4 工具链构建的。
一个直接的问题是:如何把 9 亿参数的模型塞进主机 8GB 的内存里——这部分内存由 CPU 和 GPU 共享。按每个权重 8 bit 计算,每个参数占 1 字节,意味着至少需要 9GB,而整体所需通常还要更多。部分答案在于 4 bit(“Q4”)权重,其占用减半;不过每层中最大的两个矩阵使用的是 3 bit(“Q3”)权重,含缩放系数约为 3.5 bit。
采用这种组合后占用约 4.5GiB,而如果全部使用 4-bit 则约为 4.9GiB。注意力缓存(即 KV 缓存)同样是 4-bit,这使得 16K Token 的上下文窗口能够放下,相比之下该模型原生支持 256K。Parakeet 听写模型额外占用 0.33GiB,Chatterbox 语音模型分别占用 0.29GiB 和 0.09GiB,总计 5.23GiB,即 5.62GB,与 5.66GB 的软件包体积十分接近。
下一个问题是性能。Qwen3.5 带有多头预测(MTP)头,Garlic LLM 用它最多预先草拟两个 Token,再由完整模型进行验证。这与该模型训练时的设定一致,阿里巴巴模型卡上的 vLLM 使用说明也采用两个推测 Token。该项目将 2013 年硬件上的对话速度归功于此。
在这套配置下,代码生成也更快,最高可达每秒 43 个 Token。文中还详细说明了默认 temperature(即模型输出的随机性)为 0.7——阿里巴巴建议通用任务在关闭思考模式下用 0.7、开启时用 1.0——不过演示视频中使用的是 0,因此输出可预测。在每秒 35 个 Token(所报告性能区间的中位值)下,300 Token 的回答耗时不到 9 秒。作为对比,PS5 运行体积大三倍的 Qwen3.8-27B 约为每秒 20 个 Token,而更小的 Gemma 4 E4B 和 Llama 3.2 3B 分别为 61 和 124,可能更快。
PS4 Pro“尚未测试”,但开发者正在 GitHub 上向用户征集测试报告。PS5 不受支持,因为该应用是专门针对 PS4 的 GPU 和系统软件编写的。该应用本身不含漏洞利用代码,但安装 GoldHEN 需用户自行承担风险。
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Jailbroken PS4 runs 9-billion-parameter AI model at up to 43 tokens per second)
Developer Leonardo Alegría’s open-source project Garlic LLM lets a jailbroken PS4 run the 9-billion-parameter Qwen3.5-9B AI model offline on the console’s 2013-era GPU. The app talks to the GPU directly, and the model is shrunk to about 4.5GiB to fit in the console’s memory, running at 31 to 39 tokens per second.
The project, named after the memory the GPU reads directly — its sibling memory type is “Onion” — relies on a PS4 running GoldHEN, which is a homebrew enabler for exploited consoles. Garlic LLM is a free, single 5.66GB download, available on Hugging Face and installable through the PS4’s Package Installer, with about 6GB free needed, or 12GB if the package is first copied to the console’s internal disk. The developer tested it on a base/Slim PS4 on firmware 11.52.

It’s capable of voice conversation, with spoken replies in seven languages through the Chatterbox speech model and the chat model running at about 22 tokens per second. A held controller button also lets it take dictation with Nvidia’s Parakeet TDT 0.6B v3 speech-to-text model, which supports 25 European languages. Parakeet models also power local transcription and dictation apps such as Meetily and Hex. The main model is Alibaba’s small open-weight Qwen3.5-9B, which, at the claimed 31 to 39 tokens per second in text chat — the prompt and test method are not given — is usable on the console.
The closest comparison is the PS5LM fork, which runs llama.cpp as a native app on the PS5 and reaches the console’s AMD GPU through PS5_Vulkan, a port of the RADV driver. The open-source llama.cpp is a widely used program for running language models locally, and RADV is the open-source driver that lets AMD GPUs run Vulkan, the graphics and compute API. On the PS4 there is no such driver layer for homebrew; Garlic LLM uses neither llama.cpp nor a Vulkan port, or so it would appear.
The model’s operations, such as matrix multiplies over 4-bit weights, are compute kernels written in OpenCL C and compiled with LLVM for AMD’s Graphics Core Next (GCN) architecture. The weights are kept in that “Garlic” memory, while the CPU side, in plain C, handles the tokenizer, the interface, text rendering, and audio. The pipeline runs from the prompt to the CPU tokenizer, then, for each token, a PM4 command buffer — the low-level commands a graphics driver would normally generate, which, in this case, the app builds itself — and the GCN kernels, before the token is displayed or spoken. The whole thing is built with the open-source OpenOrbis PS4 toolchain.
One immediate question is how you fit a 9-billion-parameter model in the console’s 8GB of memory, which the CPU and GPU share. At 8 bits per weight, you have one byte per parameter, meaning you’d need at least 9GB, with typically more than that required overall. Part of the answer is 4-bit (“Q4”) weights, which take half as much, although 3-bit (“Q3”) weights, at about 3.5 bits with scales, are used for the two largest matrices in each layer.
Using the combination gets you to about 4.5GiB, versus about 4.9GiB if it were all 4-bit. The attention, or KV, cache is also 4-bit, which lets a 16K-token context fit, in comparison to the model’s native 256K. The Parakeet dictation model adds 0.33GiB and the Chatterbox voice models 0.29GiB and 0.09GiB, for a grand total of 5.23GiB, or 5.62GB, which closely matches the 5.66GB package size.
The next question is performance. Qwen3.5 comes with a multi-token prediction (MTP) head, and Garlic LLM uses it to draft up to two tokens ahead, with the full model verifying. This matches what the model was trained for, and the vLLM instructions on Alibaba’s model card use two speculative tokens. The project credits this for its chat speed on 2013 hardware.
Code also comes out faster with this setup, at up to 43 tokens per second. Also detailed is the default temperature, or model variability, which is 0.7 — Alibaba recommends 0.7 for general tasks with thinking off and 1.0 with it on — although the demo video uses 0, so it is predictable. At 35 tokens per second, the middle of the reported performance range, a 300-token answer takes under nine seconds. As a comparison, the PS5 ran the three-times-larger Qwen3.8-27B at about 20 tokens per second and the smaller Gemma 4 E4B and Llama 3.2 3B at 61 and 124, respectively, which is potentially quicker.
The PS4 Pro is “not tested yet,” but the developer is asking users for reports on GitHub. The PS5 is not supported as the app is written specifically for the PS4’s GPU and system software. The app itself contains no exploit, but GoldHEN is installed at the user’s own risk.
正文保留英文原文,标题与摘要由机器翻译。
原文发布于 Tom's Hardware(Jailbroken PS4 runs 9-billion-parameter AI model at up to 43 tokens per second),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。