OpenAI 研究者称其 AI 工具价值超近半同事
OpenAI 在内部对编码智能体的使用正在快速增长,理论上的 Token 成本如今已超过研究人员的基本工资。尽管团队不愿在没有这些工具的情况下工作,但这些模型在常规任务之外仍会产生大量“AI slop”。
在第二份关于内部研究加速的报告中,OpenAI 公布了自己的数据,说明 AI 智能体在多大程度上深度嵌入新模型的开发。该公司表示,其目标是记录所谓递归自我改进(RSI)的指标,即 AI 系统加速未来几代系统的开发。
报告称,像 Codex 和 Dots 这样的内部工具的使用量近期激增。按照常规 API 标价计算,中位研究人员在 10 月每天会消耗价值 2,400 美元的 Token,较上月增长 62%。在使用量排名前 10% 的用户中,这一理论价值翻倍,达到每天 23,000 美元。
据报道,折算成八小时工作日,这些智能体相当于每位研究人员每个自然日运行 5.7 个工作日,而在预训练和后训练团队中约为 8 个工作日。超过三分之二的受追踪研究人员在高峰使用时至少并行运行四个智能体。
据 OpenAI 称,按 API 理论成本计算,9 月这些智能体的运行时长相当于研究人员常规薪资成本的两倍。不过,这一计算不包括股权部分,而股权在 OpenAI 的总薪酬中占 70% 到 80%。若以该公司的总研究算力来衡量,内部智能体的使用仅占 2%。
很大一部分 AI 使用量用于沟通和编码。| 图片:OpenAI
常规任务成功率高,代码脚手架方面存在弱点
据 OpenAI 称,3 月至 10 月期间,自主智能体在无人干预下处理 2 至 4 小时任务的成功率估计值从 23% 升至 46%。对于 4 至 8 小时的任务,该比例从 13% 攀升至 35%。不过,按 Epoch AI 的分类体系细分后可以看出,这些系统主要处理的还是简单任务。
据报道,智能体完成训练运行监控的比例为 79%,独立回答技术问题的比例为 72%,但在更复杂的工程工作上经常失败。搭建评估环境和强化学习环境的成功率为 29%,为推理和服务系统编写代码的成功率为 25%。
在一项针对 101 名员工的内部非代表性调查中,研究者给出的中位数回答是:他们宁愿放弃团队 40% 的人员或 30% 的算力,也不愿三个月没有当前的编程智能体。在另一项针对 28 名员工的快速投票中,有 27 人更倾向于最新的内部模型,而非一位处于职业中期的真人工程师。
缺乏研究品味,需要人类监督
对十名研究者的定性访谈揭示了重度使用的弊端。受访者表示,这些模型对好的研究想法以及代码和数据质量缺乏任何「品味」。智能体会迷失在不必要的步骤中,在分析中做出错误假设,并提交质量低劣的「AI 垃圾」代码改动以及无法使用的数据产物。密切的人类监督对于避免浪费算力周期至关重要。
据受访团队称,GPT-6 Astra 和 GPT-6.2 Astra 的关键概念构思完全来自人类,且集中在后训练和安全领域。只有在 GPT-6.2 Astra 的预训练中,才有研究人员表示,他们通过与 Codex 对话获得了重要洞见。
OpenAI 在结论中提醒,迄今从人机协作中收集到的数据可能无法线性外推。该公司指出,AI 近期在数学领域取得了快速突破,并认为随着模型获得更多自主性,深度学习研究本身也可能出现类似的突然进展。
评估机构 METR 此前在竞争对手 Anthropic 内部进行了为期十一周的观察后估计,AI 的使用已使那里的开发速度几乎翻倍。在此之前,OpenAI 曾在 9 月初发布的首份研究加速报告中宣称,已达到其自我设定的自动化 AI 研究实习生目标。首席科学家 Jakub Pachocki 在随附的一篇文章中警告可能失去控制,此前一项审计发现监督系统存在诸多漏洞。
不炒作,只讲 AI 新闻——由人类精选
畅读 THE DECODER 上的每一篇文章
参与评论与社区讨论
每周一封 AI 新闻回顾邮件
每年 6 次:“AI Radar”——深入探讨最重要的 AI 议题
每日 AI 新闻,始终紧跟最新动态
我们完整的十年存档
由拥有 10 年以上 AI 领域经验的团队报道
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(OpenAI's own researchers now say they value their AI tools more than nearly half their colleagues)
OpenAI's internal use of coding agents is growing fast, with theoretical token costs now exceeding researchers' base salaries. Even though teams are reluctant to work without these tools, the models still produce a lot of "AI slop" beyond routine tasks.
In a second report on internal research acceleration, OpenAI has published its own data on how deeply AI agents are embedded in the development of new models. The company says it aims to document indicators of so-called recursive self-improvement, or RSI, where AI systems speed up the development of future generations.
Usage of internal tools like Codex and Dots has surged recently, according to the report. At regular API list prices, the median researcher would have consumed $2,400 worth of tokens per day in October, up 62 percent from the previous month. Among the top 10 percent of users, that theoretical value doubled to $23,000 per day.
Converted into eight-hour workdays, the agents reportedly run the equivalent of 5.7 workdays per researcher per calendar day, and about eight in the pre- and post-training teams. More than two-thirds of tracked researchers ran at least four agents in parallel at peak usage.
The theoretical API equivalent of that agent runtime was twice as high as researchers' regular salary costs in September, according to OpenAI. That calculation excludes equity packages, though, which make up 70 to 80 percent of total compensation at OpenAI. Measured against the company's total research compute, internal agent use accounted for just 2 percent.
A large share of AI usage goes toward communication and coding. | Image: OpenAI
High success rates on routine tasks, weaknesses in code scaffolding
Between March and October, the estimated success rate of autonomous agents working without human intervention on two- to four-hour tasks rose from 23 to 46 percent, according to OpenAI. For four- to eight-hour tasks, the rate climbed from 13 to 35 percent. A breakdown using an Epoch AI taxonomy shows, however, that the systems mainly handle simple tasks.
Agents reportedly complete training-run monitoring 79 percent of the time and answer technical questions independently 72 percent of the time, but they frequently fail at more complex engineering work. The success rate for building evaluation and reinforcement-learning environments sits at 29 percent, and for writing code for inference and serving systems at 25 percent.
In an internal, non-representative survey of 101 employees, researchers said at the median that they would rather give up 40 percent of their team's staff or 30 percent of their compute capacity than go three months without current coding agents. In a separate quick poll of 28 employees, 27 preferred the latest internal model over a mid-career human engineer.
Lack of research taste requires human oversight
Qualitative interviews with ten researchers revealed the downside of heavy usage. The models lack any "taste" for good research ideas and for code and data quality, according to the interviewees. Agents get lost in unnecessary steps, make wrong assumptions in analyses, and submit subpar "AI slop" code changes along with unusable data artifacts. Close human oversight is essential to avoid wasted compute cycles.
The key conceptual ideas for GPT-6 Astra and GPT-6.2 Astra came exclusively from humans in the post-training and safety areas, according to the teams interviewed. Only in pretraining for GPT-6.2 Astra did individual researchers say they gained important insights through dialogue with Codex.
OpenAI cautions in its conclusion that the data gathered so far from human-AI collaboration may not extrapolate linearly. The company points to recent, rapid AI breakthroughs in mathematics and considers it possible that similarly abrupt advances could occur in deep-learning research itself as models gain more autonomy.
The evaluation organization METR had recently estimated, after an eleven-week look inside competitor Anthropic, that AI use there had nearly doubled the pace of development. Before that, OpenAI had declared in its first research-acceleration report in early September that it had reached its self-imposed goal of an automated AI research intern. Chief scientist Jakub Pachocki warned of losing control in an accompanying post, after an audit revealed numerous gaps in the oversight system.
AI News Without the Hype – Curated by Humans
Full access to every article on THE DECODER
Join the comments and community discussions
A weekly AI news recap via mail
6x/year: "AI Radar" — deep dives on the AI topics that matter most
Daily AI news, always up to date
Our full ten-year archive
Covered by a team with 10+ years in AI
原文为英文,正文由自动抓取并保留原文。
原文发布于 The Decoder(OpenAI's own researchers now say they value their AI tools more than nearly half their colleagues),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。