Runway 开源 Praxis-1 机器人控制模型

Runway 已发布其机器人模型 Praxis-1,采用开放权重,并以 Apache 2.0 许可开放源代码。
该模型在视频和机器人数据上训练,可处理摄像头图像、文本指令和位置数据,据此计算出相应的机器人动作。
据 Runway 称,Praxis-1 可通过额外训练适配新的机器人系统。不过,该模型偶尔会混淆外观相似的物体,而且目前还无法可靠地自行规划复杂的动作序列。
AI 公司 Runway 已发布其机器人模型 Praxis-1,开放权重和源代码。该模型利用从视频中学到的内容来控制机器人,并在公司自己的测试中展现出令人期待的结果。
Runway 已发布其用于控制机器人的 AI 模型 Praxis-1。该公司于 9 月下旬首次推出 Praxis-1,但最初仅向部分合作伙伴开放访问。伴随此次发布,Runway 还发布了一份详细的研究报告,其中包含新的性能测试。
Runway 的模型权重和源代码基于 Apache 2.0 许可提供,该许可允许商业使用。
Praxis-1 接收摄像头图像、文本指令以及有关机器人位置的信息。基于这些输入,它预测环境将如何变化,并计算出正确的机器人动作。随着新的摄像头画面输入,该模型会持续调整其控制。
Praxis-1 以视频训练为基础来控制机器人
Praxis-1 建立在一个视频模型之上,该模型从普通视频素材中学会了物体如何移动、如何相互作用。Runway 随后用人类动作视频和机器人数据对其进行了微调。这种方法有望减少对昂贵的机器人专用数据集的需求,而这类数据集传统上正是为了训练此类系统才被专门采集的。
Runway 以 Gen-4.5 等生成式视频模型而闻名。借助 Praxis-1,该公司正把其视频研究带入机器人领域。Runway 称 Praxis-1 为世界动作模型(World Action Model),它能预测场景将如何变化,并据此推导出与之匹配的机器人动作。自 9 月以来,机器人公司 Standard Bots、Noble Machines 和 Ultra 已在不同系统上测试 Praxis-1,包括一台人形机器人、一台工业机械臂和一台双臂仓储机器人。
Praxis-1 在世界动作模型中处于领先
Praxis-1 仍然需要一些专门的机器人数据。根据其研究报告,为了让 Praxis-1 适配 Franka 机械臂,Runway 使用了公开的 DROID 研究数据集中约 277 小时的录制数据。这些录制内容展示的例如是机器人抓取和移动物体。对于一台双臂机器人,该公司使用了 1,400 到 2,600 小时的训练素材。在更早的训练阶段,Runway 已经使用过 DROID 数据以及其他机器人系统的数据。相比没有经过通用视频预训练的可比模型,这种方法在总体上是否真的需要更少的机器人数据,仍是一个尚无定论的问题。
在 Nvidia 的 RoboLab-120 排行榜上,Praxis-1 的成功率为 46.2%。该基准测试用于衡量机器人模型在仿真环境中完成 120 项任务时,分拣、抓取或堆叠物体的可靠性。同样作为 World Action Model 构建的 FLUX 3 Action 在相同测试条件下得分 42.9%。这让 Praxis-1 位居 World Action Model 类别的榜首。在总排行榜上,它位列第五,排在两个 AI 智能体和两个架构不同的机器人模型之后。所有测试均由 Runway 自行完成,结果尚未经过独立验证。
在 DROID 数据上微调后,Runway 在真实的 Franka 机械臂上测试了该模型。两个模型都必须完成同样的十项任务,比如把香蕉放到盘子上或关上抽屉。每项任务尝试十次,Praxis-1 成功 88 次,而 FLUX 3 Action 成功 80 次。
Runway 还测试了 Praxis-1 向此前从未见过的机器人类型迁移的效果,所用数据来自一个带夹爪机械臂的移动机器人平台,Google 此前曾用该平台开发 RT-1。来自这台机器人的录制数据并不在 Praxis-1 此前的训练数据中。据 Runway 称,在大约 95 小时的机器人数据上微调后,该模型在仿真测试中平均成功率达到 80%。测试涵盖四类任务,包括抓取易拉罐、开关抽屉,以及把苹果放进抽屉。Google 最初的 RT-1 模型在同一基准测试中得分 52.4%。
Praxis-1 仍有明显的局限
开发者在注册并同意访问条款后,可以从 Hugging Face 下载 Praxis-1。代码已在 GitHub 上。根据 Runway 的公告,除了基础模型外,针对不同机器人的预适配版本也已提供。基础模型需要额外训练,才能适配新的机器人系统。
标准模型版本需要一块 80 GB 显存的 GPU。根据 Runway 的显存估算,压缩版本应该能装进 32 GB 显存的 RTX 5090,不过据 GitHub 文档,实际的峰值显存占用尚未测量。
根据 Runway 的研究报告,该模型有时会混淆外观相似的物体,面对含糊的指令也会吃力。对于较长的任务,需要把每一步都明确列出,因为 Praxis-1 目前还无法可靠地自行规划复杂的动作序列。
Runway 主要将 Praxis-1 视为一套快速执行机器人动作的系统。未来,它可能与把复杂任务拆解为单个步骤的通用 AI 模型协同工作。
不带炒作的 AI 新闻——由人工精选
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Runway's open-source Praxis-1 turns video AI into a robot controller)
Runway has released its robot model Praxis-1 with open weights and source code under the Apache 2.0 license.
Trained on videos and robot data, the model processes camera images, text instructions, and position data to calculate matching robot movements.
Praxis-1 can be adapted to new robot systems through additional training, according to Runway. The model does occasionally confuse similar-looking objects, though, and it can't yet reliably plan complex action sequences on its own.
AI company Runway has released its robot model Praxis-1 with open weights and source code. The model uses what it learned from video to control robots and shows promising results in the company's own tests.
Runway has released Praxis-1, its AI model for controlling robots. The company first introduced Praxis-1 in late September but initially gave access only to select partners. Alongside the release, Runway published a detailed research report with new performance tests.
Runway's model weights and source code are available under the Apache 2.0 license, which allows commercial use.
Praxis-1 takes in camera images, text instructions, and information about a robot's position. From those inputs, it predicts how the environment will change and calculates the right robot movements. As new camera frames come in, the model continuously adjusts its control.
Praxis-1 builds on video training to control robots
Praxis-1 builds on a video model that learned how objects move and interact from ordinary footage. Runway then fine-tuned it with videos of human actions and robot data. This approach could reduce the need for expensive robot-specific datasets, which have traditionally been collected just to train these kinds of systems.
Runway is best known for generative video models like Gen-4.5. With Praxis-1, the company is bringing its video research into robotics. Runway calls Praxis-1 a World Action Model that predicts how a scene will change and derives matching robot movements from those predictions. Since September, robotics companies Standard Bots, Noble Machines, and Ultra have been testing Praxis-1 on different systems, including a humanoid robot, an industrial robot arm, and a dual-arm warehouse robot.
Praxis-1 leads among World Action Models
Praxis-1 still needs some specialized robot data. To adapt it to a Franka robot arm, Runway used about 277 hours of recordings from the public DROID research dataset, according to its research report. Those recordings show robots grasping and moving objects, for example. For a dual-arm robot, the company used between 1,400 and 2,600 hours of training material. Runway had already used DROID data and data from other robot systems during earlier training stages. Whether the approach actually needs less robot data overall than comparable models without general video pretraining is still an open question.
On Nvidia's RoboLab-120 leaderboard, Praxis-1 scores a 46.2 percent success rate. The benchmark tests how reliably robot models can sort, grasp, or stack objects across 120 tasks in simulation. FLUX 3 Action, also built as a World Action Model, scores 42.9 percent under the same test conditions. That puts Praxis-1 at the top of the World Action Model category. In the overall leaderboard, it ranks fifth behind two AI agents and two robot models with different architectures. Runway ran all the tests itself, and the results haven't been independently verified yet.
After fine-tuning on DROID data, Runway tested the model on a real Franka robot arm. Both models had to complete the same ten tasks, like placing a banana on a plate or closing a drawer. With ten attempts per task, Praxis-1 succeeded 88 times, while FLUX 3 Action managed 80.
Runway also tested how well Praxis-1 transfers to a robot type it had never seen before, using data from a mobile robot platform with a gripper arm that Google had previously used to develop RT-1. Recordings from this robot weren't part of Praxis-1's earlier training. After fine-tuning on about 95 hours of robot data, the model hit an average success rate of 80 percent in simulated tests, according to Runway. The tests covered four types of tasks, including grabbing soda cans, opening and closing drawers, and placing an apple in a drawer. Google's original RT-1 model scored 52.4 percent on the same benchmark.
Praxis-1 still has clear limits
Developers can download Praxis-1 from Hugging Face after signing up and agreeing to the access terms. The code is on GitHub. According to Runway's announcement, pre-adapted versions for different robots are already available alongside the base model. The base model needs additional training before it can work with new robot systems.
The standard model version needs a GPU with 80 GB of memory. A compressed version should fit on an RTX 5090 with 32 GB of VRAM based on Runway's memory estimates, though the actual peak memory usage hasn't been measured yet according to the GitHub docs.
The model sometimes confuses similar-looking objects and struggles with vague instructions, according to Runway's research report. For longer tasks, individual steps need to be spelled out because Praxis-1 can't reliably plan complex action sequences on its own yet.
Runway sees Praxis-1 primarily as a system for fast execution of robot movements. In the future, it could work alongside general-purpose AI models that break complex tasks down into individual steps.
AI News Without the Hype – Curated by Humans
原文为英文,正文由自动抓取并保留原文。
原文发布于 The Decoder(Runway's open-source Praxis-1 turns video AI into a robot controller),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。