11-21 更新 实时
首页/ 算力市场/ 正文

更短的 AI 训练笔记可降低小设备内存需求

算力市场 来源:TechXplore 2026-11-20 23:40 0 阅读 机器翻译
更短的 AI 训练笔记可降低小设备内存需求

继续使用 Google、Facebook 或 Apple 账号,即表示你同意《使用条款》。

更短小的 AI 训练记录可降低小型设备的内存需求

由 Sadie Harley 编辑,Robert Egan 审校

本文已按照 Science X 的编辑流程与政策进行审核。编辑在确保内容可信度的同时,重点标注了以下属性:

添加到首选来源

想象一下,试图在算力非常有限的设备上直接运行 AI 模型,比如智能手机或健康可穿戴设备。它需要实时响应,而你必须要在设备端完成这些计算,这会占用大量 GB 的内存。很可能的结果就是内存不足错误。

为了让训练更高效,工程师们常常采用一种名为“参数高效微调”(PEFT)的技术。它只调整模型参数中的一小部分——参数是指模型在接收数据时学到的、用于刻画各种模式的值——其前提假设是更新越少,内存占用就越少。但根据 MIT 计算机科学与人工智能实验室(CSAIL)与哈佛大学发表在 arXiv 预印本服务器上的新研究,实际情况并非完全如此。

他们认为,问题在于模型在训练过程中会把所有的“思维笔记”都写出来。当一个 LLM 从不同数据中学习时,它会使用所谓的“中间激活存储”,实际上是在一个隐藏的地方描述其大量的计算过程。

在这些过程中,内存占用会急剧达到峰值,因此 CSAIL 的工程师开发了“低内存激活秩子空间”(Low-memory Activation Rank Subspace,LARS)方法,它把这些细节汇聚成更精简的形式,而不是把每一次计算都存下来。与最顶尖的 PEFT 技术相比,该方法在 GPU 上将 LLM 的内存占用降低超过 23%,在 CPU 上降低约 47%。

“想象一下你在听讲座,”MIT 电气工程与计算机科学(EECS)博士生、CSAIL 研究员 Irene Tenison 说,“你不是把听到的内容全部逐字记下来去总结所学,而是用几句话复述要点。这基本上就是 LARS 在做的事,帮助我们在一台小设备上运行 AI。毕竟,模型正在更高效地微调其参数,但它们所依赖的硬件存在内存限制,而我们并不会那么频繁地升级这些硬件。”

Tenison 补充说,当你希望设备端的 LLM 分析长上下文时,LARS 会很有帮助。例如,如果你戴着一块健身手表,你可能希望它分析几分钟内的心率变化。在小型设备上,由于内存限制,这很有挑战性,但 CSAIL 的方法可以让这类智能直接在您的硬件上成为可能。

未来某天,LARS 或许还能用于实时追踪工厂中的不同机组。这样一来,AI 模型就能监管工作流程,而无需依赖云端处理其全部数据。同样,这种更高效的训练方法有可能帮助机器人就地处理周围环境的信息。

节省的内存(和能源)

为了更好地了解 LARS 究竟能节省多少内存,Tenison 和她的同事在一台 8 GB 上限的小型边缘计算设备 Raspberry Pi 5 上试用了他们的方法。他们用 LARS 和一种顶级的 PEFT 方法「LoRA」来调整 LLM 的参数(也就是微调),在 128,000 的上下文长度下,他们的方法多节省了 9% 的内存。值得注意的是,节省量随上下文长度的增加而增加。

当团队让 LLM 在内存更充裕的芯片上回答更细致的问题时,研究人员注意到了一种微妙的取舍。LARS 节省的 GB 数显著多于所有基线,但准确率略低于 LoRA。

恰如其分的是,MIT CSAIL 的方法节省的 GB 最多,在需要汇总详细信息的长上下文提示上表现也最具竞争力。在所有问题上,LARS 在高性能 NVIDIA GPU 上比其领先基线多节省约 10 GB,而在性能较弱的 AMD CPU 上,它节省的内存是后者的两倍。

但 Tenison 和她的合著者后来发现,LARS 的内存效率对能耗也有连带影响。他们在 CPU 上让 LLM 处理短、中、长提示,观察到其方法的功耗和能耗比同类基线低约 22%。

「我们目前收集到的数据表明,LARS 可能是一种更可持续的 LLM 微调方法。」Tenison 说。

进一步缩小内存占用

为了最大限度地节省内存,MIT CSAIL 的方法可以与其他方法结合使用,后者针对的是训练过程的其他环节,或是模型完成提示的方式。

例如,优化注意力机制的工具(基本上就是模型关注提示词的哪些部分,或提示词在内核中如何被处理)、冻结处理数据的某些层,或对模型进行部分或完全量化的工具,都与 LARS 兼容。这是因为其他工具并不涉及模型的中间隐藏计算。

“这些方法实际上是互补的,”Tenison 说。“我们认为,如果将它们配合使用,还能节省更多能耗。”

Irene Tenison 等,Activation-Efficient Fine-Tuning via Sequence-Compressed Low-Rank Subspaces,arXiv(2026)。DOI: 10.48550/arxiv.2604.22783

期刊信息:arXiv

这篇报道背后是谁?

生命科学与生态学学士。拥有微生物学实验室背景,并在石油、天然气和可再生能源行业具备医药新闻经验。完整简介 →

数学生物学学士,创意写作硕士。游历广泛,对科学与语言有独到见解。完整简介 →

引用:Shorter AI training notes reduce memory demands on small devices(2026 年 11 月 20 日),2026 年 11 月 20 日检索自 https://techxplore.com/news/2026-11-shorter-ai-memory-demands-small.html

本文档受版权保护。除为私人学习或研究目的进行的任何合理使用外,未经书面许可,不得复制本文档的任何部分。内容仅供参考。

研究发现,压缩 AI 内存可提升准确率

新技术实现用不到四分之一兆字节的内存进行端侧训练

计算机科学家开发出使 AI 模型更高效、更可定制的方案

基于忆阻器交叉阵列的学习系统,面向可扩展且高能效的 AI

该技术使边缘设备上的 AI 能够随时间持续学习

户外 3D 追踪技术将无人机飞行路径用作“移动标尺”

自适应 AI 智能体在废水处理测试中将实验规划与机器人控制相连接

AI 防护机制失灵:篡改行为导致模拟对话中反复出现骚扰

研究人员在 AirDrop 和 Quick Share 中发现六个此前未知的漏洞

智能戒指用红外光而非皮肤电极追踪压力

强强联合:两种金属与一种墨水如何重塑印刷储能

获取即时摘要文本(Gist)

LARS 在 LLM 微调过程中压缩中间激活值,与领先的参数高效方法相比,在 GPU 上内存占用降低超过 23%,在 CPU 上降低约 47%。CPU 功耗和能耗下降约 22%。内存节省幅度随上下文长度增加而提升,但准确率略低于 LoRA。

本摘要由 LLM 自动生成。完整免责声明

译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Shorter AI training notes reduce memory demands on small devices)

By continuing with Google, Facebook or Apple, you agree to the Terms of Use.

Shorter AI training notes reduce memory demands on small devices

edited by Sadie Harley, reviewed by Robert Egan

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Add to Preferred Sources

using smartphone Credit: Unsplash/CC0 Public Domain

Imagine trying to run AI models directly on a device with very limited computing power, like a smartphone or health wearable. It needs to respond in real time and you have to do that on-device, where it eats up a lot of gigabytes (GBs). The likely result is an out-of-memory error.

To make training more efficient, engineers have often used a technique called "parameter-efficient fine-tuning" (PEFT). It tweaks only a fraction of a model's parameters—the values that capture patterns learned when taking in data—under the assumption that fewer updates mean less memory usage. But that's not exactly the case, according to new research from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and Harvard University, published on the arXiv preprint server.

The issue, they argue, is that the models write out all their mental notes during training. As an LLM is learning from different data, it uses something called "intermediate activation storage" to essentially describe its extensive calculations in a hidden place.

During these moments, memory usage peaks sharply, so CSAIL engineers developed the "Low-memory Activation Rank Subspace" (LARS) method, which pools those details into something more concise, instead of stashing every calculation. The approach reduces LLMs' memory footprint by over 23% on GPUs and about 47% on CPUs compared to top PEFT techniques.

"Imagine you're attending a lecture," says Irene Tenison, an MIT electrical engineering and computer science (EECS) Ph.D. student and CSAIL researcher. "And instead of summarizing what you've learned by transcribing it all, you recap it in a couple of sentences. That's basically what LARS is doing to help us run AI on tiny devices. After all, models are fine-tuning their parameters more efficiently, but the hardware they rely on has memory limits, which we don't upgrade as often."

Tenison adds that LARS could be helpful when you want an on-device LLM to analyze long contexts. For example, if you're wearing a fitness watch, you may want it to analyze your heart rate over the span of a few minutes. On a tiny device, this is challenging because of memory limits, but CSAIL's approach could make that kind of intelligence possible directly on your hardware.

LARS might also be useful in tracking different units in a factory in real time, one day. That way, AI models can oversee workflows without relying on the cloud to process all their data. Likewise, this more efficient training approach could potentially help a robot process its surroundings locally.

Memory (and energy) saved

To better understand how much memory LARS saves, Tenison and her colleagues tried their method on Raspberry Pi 5, a small edge computing device with an 8 GB limit. They used LARS and a top PEFT approach, "LoRA," to adjust an LLM's parameters (also called fine-tuning), where their approach saved 9% more memory at a context length of 128,000. Notably, savings increased with context length.

When the team asked LLMs to answer more detailed questions on chips with more memory available, the researchers noticed a subtle trade-off. LARS saved significantly more GBs than all baselines, but was slightly less accurate than LoRA.

Fittingly, MIT CSAIL's approach saved the most GBs and performed the most competitively on long-context prompts, where it had to summarize detailed information. Across all questions, LARS saved about 10 GB more than its leading baseline on high-performance NVIDIA GPUs, and on less powerful AMD CPUs, it saved twice as much memory.

But LARS' memory efficiency also has a downstream effect on energy consumption, as Tenison and her co-authors later found out. They tasked an LLM with short, intermediate and long prompts on a CPU and observed that their method used roughly 22% less power and energy than comparable baselines.

"The data we've collected so far shows LARS may be a more sustainable LLM fine-tuning approach," says Tenison.

Shrinking the memory footprint even further

To maximize memory savings, MIT CSAIL's method can be combined with approaches that address other parts of the training process, or the ways it completes prompts.

For instance, tools that optimize attention mechanisms (basically which parts of a prompt a model focuses on or how the prompts are processed within the kernels), freeze certain layers processing the data, or partly or fully quantize a model are compatible with LARS. This is because other tools don't address models' intermediate, hidden calculations.

"These approaches are actually complementary," says Tenison. "We think that if you use them in tandem, you'll save even more energy."

Irene Tenison et al., Activation-Efficient Fine-Tuning via Sequence-Compressed Low-Rank Subspaces, arXiv (2026). DOI: 10.48550/arxiv.2604.22783

Journal information:arXiv

Who's behind this story?

Sadie Harley

BSc Life Sciences & Ecology. Microbiology lab background with pharmaceutical news experience in oil, gas, and renewable industries. Full profile →

Bachelor's in mathematical biology, Master's in creative writing. Well-traveled with unique perspectives on science and language. Full profile →

Citation: Shorter AI training notes reduce memory demands on small devices (2026, November 20) retrieved 20 November 2026 from https://techxplore.com/news/2026-11-shorter-ai-memory-demands-small.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Shrinking AI memory boosts accuracy, study finds

New technique enables on-device training using less than a quarter of a megabyte of memory

Computer scientists develop solutions for making AI models more efficient and customizable

A memristor crossbar-based learning system for scalable and energy-efficient AI

Technique enables AI on edge devices to keep learning over time

Outdoor 3D tracking technology uses a drone's flight path as a 'moving ruler'

Adaptive AI agent connects experimental planning with robotic control in wastewater treatment test

AI guardrails falter as tampering drives repeated harassment in simulated conversations

Researcher finds six previously unknown vulnerabilities in AirDrop and Quick Share

Smart ring tracks stress with infrared light instead of skin electrodes

Better together: How two metals and one ink are reshaping printed energy storage

Get Instant Summarized Text (Gist)

LARS compresses intermediate activations during LLM fine-tuning, reducing memory use by over 23% on GPUs and about 47% on CPUs versus leading parameter-efficient methods. CPU power and energy use fell by roughly 22%. Memory savings increased with context length, with slightly lower accuracy than LoRA.

This summary was automatically generated using LLM. Full disclaimer

This site uses cookies to assist with navigation, analyse your use of our services, collect data for ads personalisation and provide content from third parties. By using our site, you acknowledge that you have read and understand our Privacy Policy and Terms of Use.

原文为英文,正文由自动抓取并保留原文。

本文由采集器自动同步

原文发布于 TechXplore(Shorter AI training notes reduce memory demands on small devices),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗