11-19 更新 实时
首页/ 深度/ 正文

清理机器人的模糊记忆

深度 来源:MIT News AI 2026-11-05 13:00 1 阅读 机器翻译
清理机器人的模糊记忆

在紧急情况下,人类某天可能会让机器人助手去取走廊尽头的壁挂式除颤器。但如果这个机器人在其存储记忆中把除颤器错误标记为灭火器,它可能就不知道去哪里找到这个救命的设备。

这款机器人助手或许能从 MIT 研究人员创建的一套新记忆系统中受益。该系统会识别机器人记忆中可能出错的条目,并修正被错误标记的物体描述。

未来,这一方法有望让家用或工业机器人更好地执行复杂的多步骤任务,比如清理餐桌,或与人类协作者一起组装一台机械。

这套名为 UQ-DAAAM(全称为 Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment,即“随时随地描述任何事物的不确定性量化”)的系统,会对比从不同视角看到的同一物体的多个描述,从而量化机器人对所见事物的不确定性。

随后,它会选择额外的视角来降低不确定性,并将这些描述合并为对该物体更准确的描述,再将其保存到记忆中。

研究人员还采用了一种特殊技术来简化这一流程,使移动机器人能够实时持续更新不确定的记忆条目。

与基线方法相比,UQ-DAAAM 提升了机器人的置信度,也提高了它回答有关物体、物体位置以及何时看到这些物体的问题时的准确率。

“与 ChatGPT 不同,当我们考虑物理 AI 时,仅仅加一句免责声明、提醒机器人有时可能会出错是不够的。让机器人知道自己不知道什么,对于这项技术安全部署并对社会产生积极影响至关重要,”MIT 航空航天系(AeroAstro)副教授、信息与决策系统实验室(LIDS)首席研究员、MIT SPARK 实验室主任、论文 UQ-DAAAM 的资深作者 Luca Carlone 表示。

与他共同完成这篇论文的还有共同第一作者、MIT 研究生 Harry Zhang 和 Nicolas Gorlo。该研究将在神经信息处理系统大会上展示。

修正记忆

这一新方法建立在研究人员此前的工作 DAAAM 之上。DAAAM 是一种机器人记忆系统,它利用视觉语言模型(VLM)对机器人通过自带摄像头拍摄到的场景中的物体进行丰富描述。这些描述以详细说明文字的形式,存储在机器人所处环境的基于 3D 地图的表示中,供其之后调用以回答问题。

但机器人有时只能部分地看到物体,或从很远处看到物体,或者其摄像头拍到的图像模糊、光线不佳。在这些情况下,DAAAM 可能会错误地描述这些物体,例如把灰色的烟囱误认成栖息在屋顶上的松鼠。

“由于无法自我纠正,这一错误记忆会在机器人继续在其环境中行进时一直存在,这可能会影响下游应用,”Zhang 说。

UQ-DAAAM 将不确定性量化集成到 DAAAM 框架中。它让机器人在环境中移动时,能够衡量自己对物体描述准确性的置信度,并修正错误的标注。

在移动过程中,机器人不断采集并存储周围世界的图像。其中许多图像从不同角度拍摄了相同的物体。UQ-DAAAM 利用这些额外的视角来识别并修正不可靠的描述。

首先,它计算 VLM 在生成用于描述可能被错误标注的物体的词语时的置信度。然后,它通过将该描述与从不同视角描述同一物体的图像标注进行对比,来衡量一致性。

“我们把这两个不确定性来源融合为一个数值,用以反映 VLM 对物体整体理解的确定程度,”Carlone 解释道。

如果该数值超过某一阈值,说明不确定性过高,UQ-DAAAM 就会利用该物体的更多图像来完善标注。

它借助一种启发式方法(即简单规则)来完成这一过程,该规则用于确定哪些额外视角能够降低机器人对该物体的不确定性。例如,它可能会选择几帧之后拍摄的一张图像,因为那张图清晰地显示了烟囱冒出的烟。

研究人员从数学上证明,这种启发式方法能够在不重新运行 VLM 的情况下找出最佳帧进行选择,从而大幅减少计算量。

“Nico 和 Harry 必须克服的关键挑战是,如何把 VLM 与不确定性量化结合起来,使其能在移动中的机器人上以不到一秒的时间运行,”Carlone 说。

自信地记忆

在确定哪些额外描述能够降低其对目标物体的不确定性之后,系统会使用一个 LLM 将它们融合为单条物体描述。为提高准确率,LLM 会给新加入的描述以及歧义更少的描述更高的权重。

这一精炼步骤能够消除机器人所处环境中约 92% 的物体的歧义,而每个物体只需额外添加两条描述。与当前最先进水平相比,它将物体描述的准确率提升了 5%。

在回答关于它所观察到的物体、在何处看到它们以及最近一次遇到它们是什么时候等问题时,UQ-DAAAM 给出的答案也比基线方法更准确。

“在整个机器人生成的地图上执行这一不确定性量化流程,能让它更好地理解自己所观察到的事物,”Gorlo 说。

接下来,研究人员希望用 UQ-DAAAM 来指导不同类型的机器人,例如移动式机械臂,并将其不确定性量化技术应用到其他领域。

此外,他们正在研究使用 UQ-DAAAM 来识别高不确定性区域,让机器人可以重新访问这些区域,以加深对环境的理解。他们还在探索将不确定性量化应用于预测机器人环境未来状态的系统。

这项研究部分得到了美国陆军研究实验室和美国海军研究办公室的支持。Carlone 目前正以亚马逊学者(Amazon Scholar)身份休假;本文描述的是在 MIT 完成的工作,与亚马逊无关。

译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Clearing up a robot’s fuzzy memory)

In an emergency, a human might someday ask a robotic assistant to fetch a wall-mounted defibrillator located down the hall. But if the robot has incorrectly labeled the defibrillator as a fire extinguisher in its stored memory, it may not know where to find the life-saving device.

This robotic assistant could benefit from a new memory system created by MIT researchers. The system identifies entries in a robot’s memory that might be wrong and corrects the descriptions of mislabeled objects. 

In the future, this method could make household or industrial robots better able to perform complex, multistep tasks, such as clearing a dining room table or working alongside human collaborators to assemble a piece of machinery. 

Called UQ-DAAAM (which stands for Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment), the system compares multiple descriptions of the same object, seen from different viewpoints, to quantify the robot’s uncertainty about what it saw. 

It then selects additional views to reduce the uncertainty and combines these captions into a more accurate description of the object, which it saves to memory.

The researchers also employed a special technique to streamline this process so a mobile robot can continually update uncertain memory entries in real time. 

When compared to baseline methods, UQ-DAAAM boosted the robot’s confidence and improved the accuracy of its answers to questions about objects, their locations, and when it saw them. 

“Contrary to ChatGPT, when we think about physical AI, it is not enough to simply put a disclaimer cautioning that sometimes the robot might be wrong. Allowing a robot to know what it doesn’t know is going to be essential for this technology to be deployed safely and have a positive impact on society,” says Luca Carlone, an associate professor in MIT’s Department of Aeronautics and Astronautics (AeroAstro), principal investigator in the Laboratory for Information and Decision Systems (LIDS), director of the MIT SPARK Laboratory, and the senior author of the paper presenting UQ-DAAAM.

He is joined on the paper by co-lead authors and MIT graduate students Harry Zhang and Nicolas Gorlo. The research will be presented at the Conference on Neural Information Processing Systems. 

Amending memory

This new method builds on the researchers’ prior work, DAAAM, a robot memory system that uses a vision-language model (VLM) to richly describe objects in scenes a robot captures with its onboard cameras. These descriptions are stored as detailed captions in a 3D map-based representation of the robot’s environment that it can call upon later to answer questions. 

But the robot sometimes sees objects only partially or from far away, or its camera might capture blurry or poorly lit images. In these cases, DAAAM might describe these objects incorrectly, such as by mistaking a gray chimney for a squirrel perched on a rooftop.

“With no way to correct itself, this wrong memory would persist as the robot proceeds in its environment, which could impact downstream applications,” Zhang says.

UQ-DAAAM integrates uncertainty quantification into the DAAAM framework. It allows the robot to measure its confidence in the accuracy of object descriptions and fix incorrect captions as it travels through an environment.

While on the move, the robot constantly collects and stores images of the world around it. Many of these images show the same objects captured from different angles. UQ-DAAAM uses these additional viewpoints to identify and correct unreliable descriptions.

First, it calculates the VLM’s confidence level in the words it generated to describe a potentially mislabeled object. Then, it measures consistency by comparing this description to captions of images that describe the same object from different vantage points.

“We fuse both sources of uncertainty into a single number that captures how certain the VLM is in its overall understanding of the object,” Carlone explains.

If that number exceeds a certain threshold, the uncertainty is too high, and UQ-DAAAM will refine the caption by using additional images of the object.

It does this using a heuristic, or simple rule, that determines which additional viewpoints will reduce the robot’s uncertainty about the object. For instance, it might choose an image captured a few frames later that clearly shows smoke emanating from the chimney. 

The researchers mathematically proved that this heuristic can identify the best frames to select without rerunning the VLM, dramatically reducing the amount of computation. 

“The key challenge Nico and Harry had to overcome is how to combine VLMs with uncertainty quantification into something that is running in a fraction of a second on a robot that is moving,” Carlone says.

Remembering with confidence

Once it has identified additional captions that will reduce its uncertainty about the object, the system uses an LLM to fuse them into a single object description. To boost accuracy, the LLM puts more weight on new captions and those with less ambiguity.

This refinement step is able to resolve ambiguities for about 92 percent of the objects in the robot’s environment, while only requiring two additional captions per object. It increased the accuracy of object descriptions by 5 percent compared to the state of the art.

UQ-DAAAM also provided more accurate answers to questions about the objects it observed, where it saw them, and when it last encountered them than baseline methods.

“Performing this uncertainty quantification procedure across the entire map the robot generates allows it to improve its understanding of what it has observed,” Gorlo says.

Next, the researchers want to use UQ-DAAAM to guide different types of robots, like mobile robotic manipulators, and apply their uncertainty quantification techniques to other domains. 

In addition, they are studying the use of UQ-DAAAM to identify areas of high uncertainty the robot could revisit to improve its understanding of the environment. They are also exploring uncertainty quantification in systems that predict the future state of a robot’s environment.

This research is supported, in part, by the U.S. Army Research Laboratory and the U.S. Office of Naval Research. Carlone is currently on sabbatical as an Amazon Scholar; this article describes work performed at MIT and is not associated with Amazon.

正文保留英文原文,标题与摘要由机器翻译。

本文由采集器自动同步

原文发布于 MIT News AI(Clearing up a robot’s fuzzy memory),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗