新通信与内存工具减少 AI 数据传输与响应延迟

由 Gaby Clark 编辑,Robert Egan 审阅
本文已按照 Science X 的编辑流程与政策进行审阅。编辑在确保内容可信度的同时,重点强调了以下属性:
添加到首选来源
如果在其他应用正在使用时 AI 内存被回收,那么恢复这部分内存会显著增加首次响应时间。图片来源:2026 版权归所有者/作者持有。Hongseung Yu 等人,原发表于 Proceedings of ACM MobiCom 2026。依据 CC BY 4.0 授权。https://doi.org/10.1145/3795866.3844495
研究人员开发出了一些技术,通过仅传输人工智能(AI)分析所需的数据来降低通信开销,并缩短用户在智能手机上使用其他应用后返回 AI 应用时必须等待响应的时间。
由首尔大学工学院电气与计算机工程系 Kyunghan Lee 教授领导的研究团队开发了 TACToC 和 mzCache:前者是一种仅选择性传输 AI 任务所需信息的通信系统,后者是一种可减少智能手机 AI 响应延迟的内存管理系统。
与现有方法相比,TACToC 将通信成本最多降低了 14.5 倍,同时把精度损失控制在 3 个百分点以内。在多个应用同时使用的环境中,与传统的基于存储的恢复方法相比,mzCache 将 AI 的首次响应时间最多缩短了 5.5 倍。
介绍这两项研究的两篇论文已被 ACM MobiCom 2026 接收,该会议是移动计算与网络领域的会议,于 10 月 26 日至 30 日在德克萨斯州奥斯汀举行。
传统任务导向通信需要为每项任务单独训练编码器和解码器,而 TACToC 使用单一编码器,只选择并传输给定任务所需的视觉特征。通过训练小型的任务 Token 即可新增任务。图片来源:2026 年版权归所有者/作者所有。Huijeong Choe 等,原发表于 ACM MobiCom 2026 会议论文集。依据 CC BY 4.0 许可协议。https://doi.org/10.1145/3795866.3796689
“TACToC”只选择性传输 AI 任务所需的信息
推理卸载(inference offloading)是指将智能手机、自动驾驶汽车或机器人拍摄的视频发送到远程服务器进行 AI 分析,从而减轻设备的计算负担。然而,传统的视频压缩是以面向人眼的视觉质量为核心设计的,因此会传输 AI 分析并不需要的信息。
任务导向通信是一种只传输 AI 任务所需信息的方法。但现有方法需要为每项任务(如图像分类或目标识别)单独训练用于提取和压缩信息的编码器,这在处理多任务或新增任务时会造成负担。
研究团队的 TACToC(Task-Adaptive Communication for Task-oriented Communications,面向任务通信的任务自适应通信)用单一编码器支持多种 AI 任务。TACToC 无需为每项任务重新训练整个编码器,而是通过修改一个包含任务信息的小型提示,有选择地识别出该任务所需的视觉特征。例如,当自动驾驶汽车用同一段摄像头画面来识别周围物体或估算其距离时,系统会针对每项任务选择并传输最重要的信息。
该系统的核心是一个“任务可描述编码器”(task-describable encoder),它结合了两种算法:相关特征估计与基于提示的任务嵌入。系统会预测每个视觉特征对 AI 任务的贡献程度,只选取所需的信息,而任务信息则用少量可训练的 Token 来表示。
当新增一个任务时,只需训练相应的 Token。被选中的信息经过压缩后发送至服务器,服务器重建的是分析所需的特征而非原始图像,从而使现有 AI 模型无需修改即可使用。
在使用公开数据集、覆盖五项任务——语义分割、姿态估计、图像描述、图像分类和深度估计——的实验中,与现有方法相比,TACToC 将通信成本降低了 1.5× 到 14.5×,同时将精度损失控制在 3 个百分点以内。即便同时处理多个任务,随着任务数量的增加,传输的数据量变化也非常小。
切换任务只需更新约 1,900 个参数,耗时约 12.5 毫秒。与现有的面向任务的通信方法相比,TACToC 实现了约三倍的训练吞吐,而内存占用仅约为其五分之一。在 NVIDIA 的 Jetson AGX Orin 边缘 AI 设备上,它每秒可处理约 21 帧。
在反映真实移动网络条件的实验中,TACToC 在保持稳定准确率的同时,相比 H.265 视频压缩将带宽占用降低了约 16.7%,相比现有的一种面向任务的通信方法降低了约 41.1%。由于该技术无需修改现有通信网络架构即可部署,有望降低自动驾驶汽车、机器人、无人机和智能监控系统的通信开销,同时还可支持下一代 6G 语义通信和物理 AI 服务中的应用。
「mzCache」通过在计算的同时回收内存来加速 AI 响应
该团队还开发了 mzCache,以解决端侧 AI(AI 模型直接运行在智能手机上)的内存瓶颈问题。端侧 AI 在隐私方面具有优势,因为数据无需发送到外部服务器,但要让 AI 记住此前的对话,模型和对话历史都必须保留在内存中。即便是轻量级 AI 模型,模型本身和对话处理也可能占用约 5–7 GB 内存。
问题出现在用户从 AI 切换到其他应用(如相机、游戏或视频应用)时。为了腾出空间,操作系统(OS)会回收此前被 AI 占用的内存。当用户返回 AI 时,数据需要恢复,响应因此出现延迟。研究团队在真实智能手机上进行的测量显示,首次响应时间从 0.8 秒增加到约 16 秒。当内存压力变得严重时,AI 可能被强制终止,此前的对话上下文也会随之丢失。
研究人员关注到,这一问题不仅源于 AI 模型本身的计算速度,也源于操作系统的内存管理。实际上,现有的内存压缩技术只能将 AI 对话历史缩减 0.2%–9.3%。
mzCache 将 AI 内存划分为小单元,使系统只需回收其他应用所需的那部分内存。当用户返回时,它同时执行内存恢复与 AI 计算。图形处理器(GPU)利用已有的数据执行计算,而中央处理器(CPU)则恢复其余数据。
利用 AI 模型按顺序从较早的层开始计算这一特点,研究人员将系统设计为从较晚的层开始逐出内存,并从较早的层开始恢复内存。他们还结合了两条恢复路径——一条将压缩后的对话历史保留在内存中,另一条则使用存储——使恢复过程能够适应已被回收的内存容量。
在 Galaxy S25+ 和 OnePlus 12 等市售智能手机上进行的实验显示,与先将数据移至存储、之后再恢复的传统方法相比,mzCache 将 AI 的首次响应时间缩短了 2.1 倍至 5.5 倍。当 AI 内存被完全回收时,mzCache 的响应速度比 Android 默认方案快至多 25.9 倍,同时还降低了恢复与计算所消耗的能量。
这一优势在一项实验中同样得到验证:用户在 10 个应用(包括社交媒体、视频、游戏和相机应用)之间交替使用约 45 分钟。在传统操作系统方案下,AI 每次都被强制终止。而采用 mzCache 后,AI 在 10 次重复测试中始终处于活跃状态,并保留了此前对话的上下文。
mzCache 以约 6000 行代码实现,可作为 Android 应用运行,无需修改内核——即操作系统的核心组件。它可用于降低与多个应用并行运行的 AI 助手的响应延迟,未来有望扩展到使用统一内存的智能眼镜、笔记本电脑和工作站等设备。
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(New communication and memory tools reduce AI data traffic and response delays)
New communication and memory tools reduce AI data traffic and response delays
New communication and memory tools reduce AI data traffic and response delays
edited by Gaby Clark, reviewed by Robert Egan
This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:
Add to Preferred Sources
If AI memory is reclaimed while other apps are in use, restoring that memory can substantially increase the time to first response. Credit: 2026 Copyright held by the owner/author(s). Hongseung Yu et al., originally published in the Proceedings of ACM MobiCom 2026. Licensed under CC BY 4.0. https://doi.org/10.1145/3795866.3844495
Researchers have developed technologies that reduce communication overhead by transmitting only the data needed for artificial intelligence (AI) analysis and shorten the time users must wait for a response when returning to an AI application after using other apps on a smartphone.
A research team led by professor Kyunghan Lee of the Department of Electrical and Computer Engineering at Seoul National University College of Engineering has developed TACToC, a communication system that selectively transmits only the information required for AI tasks, and mzCache, a memory management system that reduces response delays for smartphone AI.
TACToC reduced communication costs by up to 14.5× compared with existing methods while keeping the loss in accuracy within 3 percentage points. In environments where multiple apps are used, mzCache reduced the AI's time to first response by up to 5.5× compared with conventional storage-based recovery methods.
The two papers describing the research were accepted for ACM MobiCom 2026, a conference in mobile computing and networking, held in Austin, Texas, from Oct. 26 to 30.
Conventional task-oriented communication requires a separate encoder and decoder to be trained for each task, whereas TACToC uses a single encoder to select and transmit only the visual features required for a given task. New tasks can be added by training small task tokens. Credit: 2026 Copyright held by the owner/author(s). Huijeong Choe et al., originally published in the Proceedings of ACM MobiCom 2026. Licensed under CC BY 4.0. https://doi.org/10.1145/3795866.3796689
'TACToC' selectively transmits only the information needed for AI tasks
Inference offloading, in which video captured by smartphones, autonomous vehicles or robots is sent to a remote server for AI analysis, can reduce the computational burden on devices. Conventional video compression, however, is designed around visual quality for human viewers and therefore transmits information that is unnecessary for AI analysis.
Task-oriented communications is an approach that transmits only the information required for an AI task. Existing approaches, however, have required separate training of encoders that extract and compress information for each task, such as image classification or object recognition, creating a burden when handling multiple tasks or adding new ones.
The research team's TACToC (Task-Adaptive Communication for Task-oriented Communications) supports multiple AI tasks with a single encoder. Instead of retraining the entire encoder for each task, TACToC changes a small prompt containing information about the task to selectively identify only the visual features needed for that task. For example, when an autonomous vehicle uses the same camera footage to identify surrounding objects or estimate their distances, the system selects and transmits the information most important for each task.
At the core of the system is a "task-describable encoder" that combines two algorithms: relevant feature estimation and prompt-based task embedding. The system predicts how much each visual feature contributes to the AI task and selects only the information that is needed, while task information is represented using a small number of trainable tokens.
When a new task is added, only the corresponding tokens need to be trained. The selected information is compressed and sent to the server, which reconstructs the features required for analysis rather than the original image, allowing existing AI models to be used without modification.
In experiments using public data sets across five tasks—semantic segmentation, pose estimation, image captioning, image classification and depth estimation—TACToC reduced communication costs by 1.5× to 14.5× compared with existing methods while keeping the loss in accuracy within 3 percentage points. Even when multiple tasks were processed simultaneously, the amount of transmitted data changed very little as the number of tasks increased.
Switching tasks required updating only about 1,900 parameters and took approximately 12.5 milliseconds. Compared with existing task-oriented communication methods, TACToC achieved approximately three times higher training throughput while using about one-fifth as much memory. On NVIDIA's Jetson AGX Orin edge AI device, it processed approximately 21 frames per second.
In experiments reflecting real-world mobile network conditions, TACToC maintained stable accuracy while reducing bandwidth use by approximately 16.7% compared with H.265 video compression and approximately 41.1% compared with an existing task-oriented communication method. Because it can be deployed without modifying existing communication network architectures, the technology is expected to reduce communication overhead for autonomous vehicles, robots, drones and intelligent CCTV systems, while also supporting applications in next-generation 6G semantic communications and physical AI services.
'mzCache' speeds up AI responses by recovering memory while computing
The team also developed mzCache to address memory bottlenecks in on-device AI, where AI models run directly on smartphones. On-device AI offers advantages for privacy because data do not need to be sent to external servers, but for AI to remember previous conversations, the model and conversation history must be retained in memory. Even a compact AI model can use approximately 5–7 GB of memory for the model itself and conversation processing.
The problem arises when a user switches from AI to another app, such as a camera, game or video application. To free up space, the operating system (OS) reclaims memory that had been used by the AI. When the user returns to the AI, responses are delayed while the data are restored. Measurements conducted by the research team on actual smartphones showed that the time to first response increased from 0.8 seconds to approximately 16 seconds. When memory pressure becomes severe, the AI may be forcibly terminated, causing the previous conversation context to be lost as well.
The researchers focused on the fact that this problem stems not only from the computational speed of the AI model itself but also from the operating system's memory management. In practice, existing memory compression techniques reduced AI conversation history by only 0.2%–9.3%.
mzCache divides AI memory into small units, allowing the system to reclaim only as much memory as other apps require. When the user returns, it performs memory recovery and AI computation simultaneously. While the graphics processing unit (GPU) performs computations using data that are already available, the central processing unit (CPU) restores the remaining data.
Taking advantage of the fact that AI models perform computations sequentially from their earlier layers, the researchers designed the system to evict memory beginning with the later layers and restore it beginning with the earlier layers. They also combined two recovery paths—one that retains compressed conversation history in memory and another that uses storage—allowing recovery to adapt to the amount of memory that has been reclaimed.
Experiments on commercially available smartphones, including the Galaxy S25+ and OnePlus 12, showed that mzCache reduced the AI's time to first response by 2.1× to 5.5× compared with conventional methods that move data to storage and later restore it. When AI memory was completely reclaimed, mzCache delivered responses up to 25.9× faster than the default Android approach, while also reducing the energy consumed for recovery and computation.
The benefits were also confirmed in an experiment in which users alternated among 10 apps—including social media, video, games and camera apps—for approximately 45 minutes. Under the conventional operating system approach, the AI was forcibly terminated each time. With mzCache, however, the AI remained active throughout 10 repeated trials and preserved the context of previous conversations.
mzCache was implemented in approximately 6,000 lines of code and can run as an Android application without modifying the kernel, the core component of the operating system. It can be used to reduce response delays for AI assistants operating alongside multiple apps and is expected to expand in the future to devices such as smart glasses, laptops and workstations that use unified memory.
Regarding the TACToC research, professor Lee said, "This research presents a highly efficient machine-to-machine communication technique that selectively transmits only the information needed for various AI tasks, establishing a technological foundation for realizing network-based physical AI."
Regarding the mzCache research, he said, "This research presents a high-performance approach to operating and managing AI models that maintains both the responsiveness and conversational context of on-device AI even under limited memory." He added, "We expect it to develop into a foundation for delivering stable AI services across a wide range of devices, including smartphones and smart glasses, and to make a significant contribution to establishing Korea as a leader in physical AI."
The TACToC research was conducted by Huijeong Choe, the first author, along with Minjae Lee, Taehun Kim and Dr. Wooseung Nam of KAIST. The mzCache research involved Hongseung Yu, a master's student and the first author, along with Minsung Kim, Jongseok Park and others.
Professor Lee served as the corresponding author of both papers. The research team plans to expand the technologies to environments in which multiple AI applications run simultaneously and to a broader range of mobile processors, including neural processing units (NPUs).
Hongseung Yu et al., mzCache: On-Device LLM Memory Management under Multitasking, Proceedings of the 32nd Annual International Conference on Mobile Computing and Networking (2026). DOI: 10.1145/3795866.3844495
Huijeong Choe et al., TACToC: Enabling Task-Adaptive Communication System with a Task-Describable Encoder, Proceedings of the 32nd Annual International Conference on Mobile Computing and Networking (2026). DOI: 10.1145/3795866.3796689
Who's behind this story?
MA in English, copy editor since 2021 with experience in higher education and health content. Dedicated to trustworthy science news. Full profile →
Bachelor's in mathematical biology, Master's in creative writing. Well-traveled with unique perspectives on science and language. Full profile →
Citation: New communication and memory tools reduce AI data traffic and response delays (2026, November 5) retrieved 6 November 2026 from https://techxplore.com/news/2026-11-communication-memory-tools-ai-traffic.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
Get Instant Summarized Text (Gist)
TACToC selectively transmits task-relevant visual features, reducing communication costs by up to 14.5-fold with accuracy loss within 3 percentage points. mzCache overlaps memory recovery with computation, reducing smartphone AI time to first response by up to 5.5-fold versus storage-based recovery while preserving conversation context.
This summary was automatically generated using LLM. Full disclaimer
This site uses cookies to assist with navigation, analyse your use of our services, collect data for ads personalisation and provide content from third parties. By using our site, you acknowledge that you have read and understand our Privacy Policy and Terms of Use.
原文为英文,正文由自动抓取并保留原文。
原文发布于 TechXplore(New communication and memory tools reduce AI data traffic and response delays),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。
