11-19 更新 实时
首页/ 技术栈/ 正文

新 LoRA 适配器将 Qwen3.8 编程成功率从 39% 提升至 65%

技术栈 来源:MarkTechPost 2026-11-19 00:07 0 阅读 机器翻译

Hugging Face 的 harness-hub 团队发布了 Qwen3.8-27B-pi-adapted,这是为 Qwen3.8-27B 打造的 LoRA 适配器。它通过强化学习训练,以便在 pi 编码智能体内部运行。Qwen3.8-27B-pi-adapted LoRA 适配器面向单一 harness,而非通用对话。对于运行本地编码智能体的开发者来说,这正是关键所在。你保留开放的基础模型,再换入一个小型、针对特定 harness 的适配器。

Qwen3.8-27B-pi-adapted 是一个 PEFT LoRA 适配器,可把 Qwen3.8-27B 转变为针对 pi 调优的模型。pi 是一个 MIT 许可的智能体工具包,带有编码智能体 CLI。该适配器以 Apache-2.0 许可发布。它作用于 q、k、v、o、gate、up 和 down 投影。它仅适用于 Qwen3.8-27B。

基础模型是一个稠密混合模型。它在 64 层中交替使用 Gated DeltaNet 线性注意力模块与 Gated Attention 模块。

该方法来自 Hugging Face 11 月 6 日发布的 harness adapters 文章。其思路是每个模型和 harness 配 1 个 LoRA 适配器。训练在未经修改的 harness 内、在可验证任务上进行。

对于这个 checkpoint,模型卡列出了以下细节:

每一次工具调用都要经过 safeagents,这是一个基于规则的代理。它会拦截诸如从 Hub 获取基准测试答案之类的调用。被拦截的 rollout 得分为 0,因此强化学习会推动模型远离奖励黑客行为。

以下所有数字均为自我报告。它们使用 pi harness,每个任务尝试 1 次。

该适配器解决了 15 个新的 SWE-bench Pro 任务,同时丢失了 2 个。在 Terminal-Bench 2.1 上,它新增解决 16 个,丢失 2 个。

效率才是更值得关注的看点。在两个模型都能解决的任务上,SWE-bench Pro 的中位轮次从 59 降到 23。在 Terminal-Bench 2.1 上,轮次从 18 降到 8。开启思考时,该适配器生成的 Token 数也比基础模型少约 4.5 倍。

开启思考时,基础模型在 SWE-bench Pro 上已经能拿到 64.7%,在 Terminal-Bench 2.1 上为 54.2%。因此,该适配器无需付出长推理链的代价,就能达到思考模式级别的准确率。开启思考时,适配器分别达到 66.7% 和 56.6%。

OpenCode 适配器的数据使用了不同的测试框架,不能直接比较。

模型卡给出了 4 条路径:

由于适配器共享同一个基础模型,单台服务器可以同时托管 pi 和 OpenCode 适配器。

可以查看模型卡、GGUF 适配器、方法论文章和 TRL 训练脚本。你也可以通过我们的 GPULeaderboard.com Chrome 扩展在这里查看部署方案和规格。

需要与我们合作推广你的 GitHub 仓库、Hugging Face 页面、产品发布或网络研讨会等?欢迎联系我们

《新 LoRA 适配器将 Qwen3.8 编码成功率从 39% 提升到 65%》一文首发于 MarkTechPost。

译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(New LoRA Adapter Lifts Qwen3.8 Coding Success From 39% to 65%)

Hugging Face’s harness-hub team has released Qwen3.8-27B-pi-adapted, a LoRA adapter for Qwen3.8-27B. It was trained with reinforcement learning to work inside the pi coding agent. The Qwen3.8-27B-pi-adapted LoRA adapter targets one harness instead of general chat. For developers running local coding agents, that is the point. You keep the open base model and swap in a small, harness-specific adapter.

TL;DR

  • Size: LoRA rank 32 (alpha 64) on Qwen3.8-27B, a 27B dense hybrid with 262,144 native context. Merged BF16 checkpoint is about 54 GB.
  • Runs on: vLLM with LoRA enabled, or llama.cpp with a GGUF adapter on a Q8_0 base. Inference VRAM is not disclosed. Training used 2x NVIDIA B200.
  • Performance: Large pass@1 gains inside pi with thinking off, plus far shorter sessions.
  • Best: 64.7% on SWE-bench Pro (hard 51), up from 39.2%.
  • Worst: 55.8% on Terminal-Bench 2.1. The base model with thinking already reaches 64.7% on SWE-bench Pro.
  • Bottom line: Best: thinking-level accuracy at about 4.5x fewer tokens. Worst: small eval sets, self-reported, pi only.

What is Qwen3.8-27B-pi-adapted?

Qwen3.8-27B-pi-adapted is a PEFT LoRA adapter that turns Qwen3.8-27B into a model tuned for pi. pi is an MIT-licensed agent toolkit with a coding agent CLI. The adapter is released under Apache-2.0. It applies to the q, k, v, o, gate, up and down projections. It only works on Qwen3.8-27B.

The base model is a dense hybrid. It alternates Gated DeltaNet linear-attention blocks with Gated Attention blocks across 64 layers.

How does the harness adapter training work?

The method comes from Hugging Face’s harness adapters post from November 6. The idea is 1 LoRA adapter per model and harness. Training runs inside the unmodified harness on verifiable tasks.

For this checkpoint, the model card lists these details:

  • Algorithm: AsyncGRPO from TRL, 8 rollouts per task at temperature 0.8.
  • Tasks: 750 scale800 tasks interleaved with 300 TermGrade tasks.
  • Reward: verifier correctness plus a tool-efficiency bonus with weight 0.1.
  • Budget: up to 50 agent steps and 1,800 seconds per rollout, thinking off.
  • Checkpoint: step 160. The run stopped at step 191 of 1,000 planned.

Every tool call passes through safeagents, a rule-based proxy. It blocks calls such as fetching benchmark answers from the Hub. Blocked rollouts score 0, so RL pushes the model away from reward hacking.

Benchmarks: how much does the LoRA adapter help?

All numbers below are self-reported. They use the pi harness, 1 attempt per task.

Benchmark (pi, thinking off)Base Qwen3.8-27Bpi-adaptedChange
SWE-bench Pro (hard 51)39.2%64.7%+25.5 ± 14.2 pts (p = 0.0023)
Terminal-Bench 2.1 (86 tasks)39.5%55.8%+16.3 ± 9.1 pts (p = 0.0013)

The adapter solved 15 new SWE-bench Pro tasks and lost 2. On Terminal-Bench 2.1, it gained 16 and lost 2.

Efficiency is the larger story. On tasks both models solved, median turns fell from 59 to 23 on SWE-bench Pro. On Terminal-Bench 2.1, they fell from 18 to 8. The adapter also generates about 4.5x fewer tokens than the base model with thinking.

How does it compare with thinking mode and other adapters?

With thinking on, the base model already scores 64.7% on SWE-bench Pro and 54.2% on Terminal-Bench 2.1. So the adapter matches thinking-level accuracy without paying for long reasoning traces. With thinking on, the adapter reaches 66.7% and 56.6%.

FeatureQwen3.8-27B-pi-adaptedQwen3.8-27B base, thinking onQwen3.8-27B-opencode-adaptedbytkim/Qwen3.8-27B-pi
TypeLoRA r=32Base modelLoRA r=32Full fine-tune, 28B
MethodAsyncGRPO RLNoneAsyncGRPO RLSFT then GRPO
HarnesspipiOpenCodepi
LicenseApache-2.0Apache-2.0Apache-2.0Apache-2.0
SWE-bench Pro (hard 51)64.7%64.7%51.0% (OpenCode)Not disclosed
Terminal-Bench 2.155.8%54.2%53.0% (OpenCode)Not disclosed
Token use~4.5x fewer than base + thinkingBaseline~6x to 8x fewer than base + thinking~41% fewer output tokens (Terminal-Bench)
Training hardware2x B200Not disclosed2x B200Not disclosed

OpenCode adapter numbers use a different harness and are not directly comparable.

How do you deploy it?

The model card gives 4 routes:

  • vLLM: serve Qwen/Qwen3.8-27B with --enable-lora --max-lora-rank 32, thinking disabled. Point pi at http://localhost:8000/v1.
  • llama.cpp: load the GGUF adapter on a Q8_0 base GGUF.
  • Llama.app on macOS: register the adapter with a 32,768-token context.
  • Merge: merge with PEFT on CPU, then serve the 54 GB BF16 result without LoRA flags.

Because adapters share 1 base model, a single server can host pi and OpenCode adapters side by side.

Key Takeaways

  • Qwen3.8-27B-pi-adapted is an Apache-2.0, rank-32 RL LoRA for the pi agent.
  • SWE-bench Pro hard-51 rises from 39.2% to 64.7%, thinking off.
  • Median turns drop from 59 to 23 on SWE-bench Pro.
  • It matches base-with-thinking accuracy at about 4.5x fewer tokens.
  • Results are self-reported on small sets; the tweet’s 29.4% baseline was corrected.


Check out the model card, GGUF adapter, methodology post and TRL training script. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.

Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

The post New LoRA Adapter Lifts Qwen3.8 Coding Success From 39% to 65% appeared first on MarkTechPost.

正文保留英文原文,标题与摘要由机器翻译。

本文由采集器自动同步

原文发布于 MarkTechPost(New LoRA Adapter Lifts Qwen3.8 Coding Success From 39% to 65%),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗