Black Forest Labs:构建视觉智能

本期节目由 Deepgram 赞助。试试用 Deepgram 打造真正理解你上下文的语音智能体!#contextiseverything
没多少人知道,我之所以全职投入 AI,正是因为 Stable Diffusion(我给公司取名 Multiverse 而不是 Metaverse,恰是为了点出扎克伯格那笔投资押错了地方)。结果发现,Dustin Podell 也有同样的顿悟时刻——模型一发布,他就辞掉了本职工作。但我们走的路截然不同……我坚持做自己熟悉的人类内容创作,而 Dustin 则与 Latent Diffusion 的共同发明人 Andreas Blattmann,以及 Stability AI 其他世界顶尖的图像生成团队一起,深度投入到 AI 内容创作中。Stability 分崩离析,但 Anj Midha 帮助核心团队拆分出来,成立了 BFL,后者随即打造出 FLUX 系列这一惊艳的新图像生成模型。可他们如今为何要进军视频和世界模型?
Andreas 举了一个用一只手就能试的例子。如果有人把球抛给你,你的手会在球到达之前就开始移动。你做得到这件事,却写不出这条轨迹。当他说打造视觉智能时,指的就是这类知识。一张静态图像能告诉你球在哪里,视频则展示它是如何到达那里的。撞击的声音能告诉你它何时撞上了东西,即便摄像机没拍到接触的瞬间。这就是 BFL 主张把三种信号合在一起训练 FLUX 3 的理由。BFL 发布 FLUX 3
《运动的马》是我们在开场中展示的 FLUX 3 示例之一。那些音频示例值得开着声音播放。FLUX 3 视频
假设摄像机从一个人正前方开始,绕着他转一圈。到某个时刻,模型就必须把此人的后脑勺生成出来。Andreas 认为,普通的二维视频中包含了大量证据,说明物体从其他角度看是什么样子。这并不能解决模型是否“理解物理”的问题——这一点我们反复争论过。但它确实提供了一个比“问某一帧看起来是否逼真”好得多的测试。
节目后段有一个很好的工程案例。如果独立地对一段视频的每一帧做锐化,每一帧可能看起来都非常出色。但按下播放键,那些额外的细节可能会闪烁,因为上采样器每次都做出了略有不同的决定。BFL 的视频上采样器源自同一个 FLUX 3 主干网络。即便用户要的只是一个更高分辨率的文件,时序一致性依然很重要。
我向他们提出的问题之一是,自回归是否正在接管图像生成。Andreas 也给了另一方应有的公允评价。自回归模型可以一次性从训练样本中的所有 Token 中学习。而在扩散这一边,团队可以把漫长的采样过程蒸馏到几步。哪种优势更重要,取决于你是在为训练模型付费,还是在等一张图生成完。
Dustin 早期的 Kontext 实验很好地说明了他们如何决定把资源投在哪里。他怀疑,一个能画出戴帽子的狗的图像模型,其实已经对帽子该戴在哪里有所了解。他和一位同事收集了一小批标注了关系的照片。一次几个小时的训练就让模型学会了把 logo 印在衬衫上。效果很粗糙,但足以拿到下一次外出会议上展示。随后团队在数据和模型上投入了几个月。FLUX Kontext 示例
我最喜欢的用户故事出现在接近结尾的地方。人们把老旧、破损的家庭照片输入 FLUX 进行修复,然后把照片拿给家人看。Dustin 说他没有预料到这种用法。在谈过如何在一次次编辑之间、跨帧之间保持某个人的身份一致之后,这是一个很动人的例子,说明如果你真做对了,究竟是谁会注意到。
BFL 解释了静态图像、运动镜头和配乐在训练过程中各自贡献了什么。FLUX 3 发布与研究概览
观看并聆听同时生成视频与声音的结果。FLUX 3 视频演示
看看该实验室如何教会模型在按需求做出修改的同时保留图像的其余部分。FLUX Kontext 示例
Andreas Blattmann 共同创造了 Stable Diffusion,并共同发明了其背后的潜在扩散方法。嘉宾简介 · 原始研究
Dustin Podell 领导了 FLUX Kontext,并且是 SDXL 的第一作者。Dustin 的项目贡献 · FLUX Kontext · SDXL
第一篇 Latent Space 文章源于 Stable Diffusion 的发布。多元宇宙,而非元宇宙
0:00 主持人介绍 3:39 冷开场 4:23 黑森林渊源 9:23 Dustin 的历程 17:04 视觉智能 27:38 深入 SDXL 33:59 学习运动 39:57 流匹配 46:57 创立 BFL 50:36 Deepgram 赞助商插播 51:43 FLUX 的早期应用场景 54:58 用 Kontext 进行编辑 1:03:15 训练数据 1:07:32 测试一次编辑 1:13:12 自回归 1:20:50 世界动作模型 1:30:08 2027 年的机器人技术 1:32:16 谁在使用 FLUX?
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Building Visual Intelligence — Andreas Blattmann & Dustin Podell, Black Forest Labs)
This episode is sponsored by Deepgram. Try Deepgram for voice agents that truly understand your context! #contextiseverything
Not many people know that Stable Diffusion is why I went fulltime into AI (I aptly titled Multiverse, not Metaverse to call out Zuck’s mistaken investment). It turns out that Dustin Podell also had the same aha moment - quitting his day job when the model came out. But we took very different paths… I stuck to what I knew in human content creation, and Dustin immersed deep into AI content creation with Andreas Blattmann, co-inventor of Latent Diffusion and the rest of the world-leading imagegen crew at Stability AI. Stability fell apart, but Anj Midha helped the core group spin out to BFL, which immediately built an incredible new imagegen model in the Flux series. But why are they now going into video and world models?
Andreas gives an example you can try with one hand. If somebody tosses you a ball, your hand starts moving before the ball reaches you. You can do that without being able to write down the trajectory. When he talks about building visual intelligence, that’s the kind of knowledge he means. A still can show you where the ball is. A video shows you how it got there. The sound of impact can tell you when it hit something, even if the camera missed the contact. That’s BFL’s argument for training FLUX 3 on the three signals together. BFL’s FLUX 3 release

The horse in motion is one of the FLUX 3 examples we show in the intro. The audio examples are worth playing with the sound on. FLUX 3 Video
What a moving camera teaches you
Suppose the camera starts in front of a person and pans around them. At some point the model owes you the back of their head. Andreas thinks ordinary two-dimensional video contains a great deal of evidence about what objects look like from other angles. That doesn’t settle whether a model “understands physics,” a point we go back and forth on. It does give you a much better test than asking whether one frame looks convincing.
There’s a good engineering example late in the episode. Sharpen each frame of a clip independently and each frame might look terrific. Hit play and the extra detail can shimmer, because the upscaler made a slightly different decision every time. BFL derives its video upscaler from the same FLUX 3 backbone. Temporal consistency matters even when all the user asked for was a higher-resolution file.
What’s worth spending training compute on?
One of the questions I had for them was whether autoregression was taking over image generation. Andreas gives the other side its due. An autoregressive model can learn from all the tokens in a training example at once. On the diffusion side, the team can distill a long sampling process down to a handful of steps. Which advantage matters depends on whether you’re paying to train the model or waiting for an image to finish.
Dustin’s early Kontext experiment is a nice example of how they decide where to invest. He suspected an image model that could draw a dog wearing a hat already knew something about where hats go. He and a colleague collected a small set of photos labeled with relationships. A training run of a few hours got the model to put a logo on a shirt. It was rough, but it was enough to take to the next offsite. The team then spent months on the data and the model. FLUX Kontext examples
My favorite user story comes near the end. People were feeding FLUX old, degraded family photos, restoring them, then showing the pictures to their families. Dustin says he hadn’t predicted that use. After talking about preserving somebody’s identity across edits and across frames, it’s a lovely example of who actually notices if you get it right.
Show notes
BFL explains what a still image, a moving camera and a soundtrack each contribute during training. FLUX 3 launch and research overview
Watch and listen to the results of generating video and sound together. FLUX 3 Video demos
See how the lab teaches a model to make a requested change while preserving the rest of an image. FLUX Kontext examples
Andreas Blattmann co-created Stable Diffusion and co-invented the latent diffusion method behind it. Guest bio · Original research
Dustin Podell led FLUX Kontext and was the lead author of SDXL. Dustin’s project credits · FLUX Kontext · SDXL
The first Latent Space essay grew out of the Stable Diffusion release. Multiverse, not Metaverse
Chapters
0:00 Host introduction
3:39 Cold open
4:23 Black Forest roots
9:23 Dustin’s path
17:04 Visual intelligence
27:38 Inside SDXL
33:59 Learning motion
39:57 Flow matching
46:57 Starting BFL
50:36 Deepgram sponsor break
51:43 Early use cases for FLUX
54:58 Editing with Kontext
1:03:15 Training data
1:07:32 Testing an edit
1:13:12 Autoregression
1:20:50 World action models
1:30:08 Robotics in 2027
1:32:16 Who uses FLUX?
正文保留英文原文,标题与摘要由机器翻译。
原文发布于 Latent Space(Building Visual Intelligence — Andreas Blattmann & Dustin Podell, Black Forest Labs),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。