11-19 更新 实时
首页/ 技术栈/ 正文

为高效长上下文推理协同设计稀疏注意力

技术栈 来源:NVIDIA Developer 2026-11-17 08:45 0 阅读 机器翻译
为高效长上下文推理协同设计稀疏注意力

智能体式工作负载、长文档和大型代码库正将上下文长度从数万 Token 推向数十万 Token。在这样的长度下,注意力机制成为 Transformer 层中开销最大的部分。稠密注意力会将每个 query 与每个 key 进行评分,因此其每个 query 的成本随上下文长度 N 增长,而在 128K Token 时,它可能占据预填充时间的大部分,在……

译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Co-Designing Sparse Attention for Efficient Long-Context Inference)
Agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousands. At those lengths,...

Agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousands. At those lengths, attention becomes the most expensive part of a transformer layer. Dense attention scores every query against every key, so its cost per query grows with the context length N, and at 128K tokens it can take most of the prefill time in a…

Source

正文保留英文原文,标题与摘要由机器翻译。

本文由采集器自动同步

原文发布于 NVIDIA Developer(Co-Designing Sparse Attention for Efficient Long-Context Inference),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗