AI 让光计算迎来第二次机会

面向技术内行人士
面向技术内行人士
IEEE Spectrum 是 IEEE 的旗舰出版物——IEEE 是全球最大的致力于工程与应用科学的专业组织。我们的文章、视频和信息图表让读者了解技术、工程与科学领域的最新进展。
创建账户,即可享受更多免费内容和权益
将文章保存以便日后阅读需要 IEEE Spectrum 账户
The Institute 的内容仅会员可访问
下载完整 PDF 期刊为 IEEE 会员专属权益
下载本电子书为 IEEE 会员专属权益
《Spectrum》数字版仅限 IEEE 会员访问
关注相关话题是 IEEE 会员专属功能
加入全球最大的工程与应用科学专业组织,即可访问《Spectrum》的全部文章、历史存档、PDF 下载及其他权益。进一步了解 IEEE →
加入全球最大的工程与应用科学专业组织,即可获取这本电子书,以及 IEEE Spectrum 的全部文章、历史存档、PDF 下载及其他权益。进一步了解 IEEE →
畅享数千篇文章——完全免费
创建账户即可获得专属内容与功能:收藏文章、下载合集、发表评论——全部免费!如需完整访问权限与全部权益,请订阅《Spectrum》。
AI 让光计算迎来又一次机会
降低 AI 推理能耗的努力,可能带来意想不到的回报
数据中心已经在网络中使用光。初创公司 Optalysis 旨在利用专门设计的波导,为网络路径增加计算能力,在数据传输过程中完成数学运算。
用光计算的想法已经存在 70 多年,但直到最近,它基本上仍停留在研究领域。不过,过去几年里,光计算已发展成为一个快速增长的产业:该领域公司的市场估值如今总计 22.2 亿美元,高于 2022 年的 4 亿美元。有一项预测认为,未来几年该市场将继续呈指数级增长。
这一突然激增的原因不会让任何人意外:AI。尤其是 AI 推理——使用 AI 模型响应用户查询的过程——推动了大量庞大且高耗能数据中心的建设,而相对节能的光学系统被视为一种潜在的前进方向。
“对[光子学]的大部分兴趣,源于 AI 推理实际上已经受到功耗限制,而且是以一种相当惊人的方式,”位于纽约州伊萨卡的康奈尔大学应用与工程物理系副教授 Peter McMahon 表示。他研究光计算已超过十年。
至少从纸面上看,光学有望以远低于当今 GPU 的功耗完成同样的数学运算。从电子转向光学将是一种相当激进的解决方案。尽管如此,光学计算初创公司正像雨后春笋般在美国、中国、德国和以色列涌现。虽然与 NVIDIA 这样的 GPU 制造商竞争是个雄心勃勃的目标,但对光学计算的投资已经推动了互补性方法的发展,比如设计替代性的 AI 算法、承接科学计算任务,以及增强网络内计算。
总部位于英国的 Optalysys 正把光学计算元件插入网络路径——数据中心在那里已经在使用光。该公司的基本片上光学元件(如插图所示)是一个傅里叶变换计算器。Optalysys
“我认为,如果这些公司中有很多都能在这条路上做出不错的尝试,对这项技术来说是好事,”耶鲁大学应用物理学助理教授 Logan Wright 说。他位于康涅狄格州纽黑文,曾是 McMahon 实验室的博士后。“我认为,由此开发出的技术将会非常、非常有用,并能催生一些光子学系统——它们也许赶不上这波 AI 浪潮,但能够做出相当特别的事情。”
光学能拯救推理吗?
推理占 AI 能耗的大头。业界已经在花数十亿美元设计先进 GPU 和其他芯片,以尽可能高效地运行推理。推理计算的核心是一个运算,每秒要执行无数次:矩阵-向量乘法。
早在 1978 年,光学计算先驱 Joseph W. Goodman 及其同事就提出了一种用光进行矩阵-向量乘法的方法。该装置相对简单。假设一个长度为 N 的向量需要与一个 N×N 的矩阵相乘。N 个发光二极管排成一排,将向量的元素编码为光的强度——单个二极管发出的光越亮,该二极管编码的值就越大。然后,一个柱面透镜将光垂直展开,本质上就是把该向量复制 N 次。接着,光穿过一个 N×N 像素的衰减器。在每个像素处,光会根据矩阵中该位置对应的值被衰减。让光通过衰减器就完成了乘法。最后,一个与第一个透镜垂直放置的柱面透镜重新收集每一行的光,并将其求和。最终向量由一排探测器测量。
向量-矩阵乘法
当今常见 AI 中计算开销最大的部分是向量-矩阵(或矩阵-矩阵)乘法。对于长度为 N 的向量(这里 N = 3),它需要 N²(这里为 9)次乘法步骤和三次累加步骤——同时进行多次加法。
关于光学为何可能比电子学更具优势,其论证大致如下:将一个大小为 N×N 的矩阵乘以一个长度为 N 的向量,需要 N2 次乘法步骤。在电子学中,每一次乘法都会消耗一定能量,因此总能耗按 N2 规模增长。然而在光学中,乘法步骤是被动的:只要矩阵保持不变,让光通过衰减器几乎不消耗能量。唯一的开销在于将向量编码为光以及读出结果,但向量的长度是 N,而不是 N2。因此,随着所乘的向量和矩阵规模变大,能耗将按 N 而非 N2 的规模增长。
自 Goodman 于 1978 年提出这一设想以来,可用的光学元件质量已大幅提升。事实上,2022 年,由 McMahon 领导的康奈尔大学研究团队展示了一个光学神经网络,其每次乘法运算的能耗不到一个光子——约 2.5 x 10-19 焦耳。在标准计算机中,单次乘法运算的能耗可能在 10-14 到 10-12 焦耳之间,这表明光学有可能将能耗降低多达 100,000 倍。
但 McMahon 提醒不要做这样的外推。首先,将数据编码为光学的开销并未计入这一计算。即便以光来实现时规模扩展性更有利,在实践中重要的不只是规模扩展,还有系数。假设用电子学做一次乘法的能耗是 1 皮焦耳,而用光学编码一个向量元素的能耗是 1,000 皮焦耳。那么当向量的长度 N 少于 1,000 个元素时,电子学方案仍然胜出。
McMahon 表示,这个例子与现实相差不远:“关键问题之一是,你要把系统做多大才能开始取胜?很多人针对各种类型的神经网络做过这类计算。一个非常粗略的答案是,介于 N=1,000 到 N=10,000 之间。”
同时,还有一条名为阿姆达尔定律(Amdahl's law)的计算原理也值得牢记。
“阿姆达尔定律甚至不该被称为定律。它只是一条简单的常识,”McMahon 说。“如果你有某种能加速计算的东西,它可以加速 90% 的代码,但还有 10% 的代码无法被它加速,那么你能获得的最大加速比就是 10 倍。”
在 AI 推理中,重点在于节省能耗,而非节省时间。但阿姆达尔的观察依然适用。当今的 AI 推理代码中有多大比例能更好地由光学来处理,这是一个复杂的问题,因具体方法而异。但这个比例不会是 100%。矩阵—向量乘法,或者说矩阵—矩阵乘法,只是推理工作负载的一部分——数据上还需要执行其他非线性运算。
综合考虑所有这些限制条件,如今的光计算公司把目标定在相较当前 GPU 实现 10 到 80 倍的能效提升。问题在于,它们能否在传统电子技术追平这些收益之前,以规模化方式兑现这一承诺。
弥合自由空间中的鸿沟
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(AI Is Giving Optical Computing Another Shot)
FOR THE TECHNOLOGY INSIDER
FOR THE TECHNOLOGY INSIDER
IEEE Spectrum is the flagship publication of the IEEE — the world’s largest professional organization devoted to engineering and applied sciences. Our articles, videos, and infographics inform our readers about developments in technology, engineering, and science.
Enjoy more free content and benefits by creating an account
Saving articles to read later requires an IEEE Spectrum account
The Institute content is only available for members
Downloading full PDF issues is exclusive for IEEE Members
Downloading this e-book is exclusive for IEEE Members
Access to Spectrum 's Digital Edition is exclusive for IEEE Members
Following topics is a feature exclusive for IEEE Members
Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more about IEEE →
Join the world’s largest professional organization devoted to engineering and applied sciences and get access to this e-book plus all of IEEE Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more about IEEE →
Access Thousands of Articles — Completely Free
Create an account and get exclusive content and features: Save articles, download collections, and post comments — all free! For full access and benefits, subscribe to Spectrum .
AI Is Giving Optical Computing Another Shot
The push to cut AI inference’s energy use could pay off in unexpected ways

Data centers already use light in networking. Startup Optalysis aims to use specially engineered waveguides to add computational capacity to the networking path, doing math while data is in transit.
The idea of computing with light has been around for over 70 years, but until recently it’s remained largely in the realm of research. Over the past few years, though, optical computing has emerged as a fast-growing industry: Market valuation of companies in this sector now totals US $2.22 billion, up from $0.4 billion in 2022. One projection suggests that the market will continue to grow exponentially in the coming years.
The reason for this sudden surge will surprise no one: AI. In particular, AI inference—the process of using AI models to respond to user queries—has driven much of the construction of huge, power-hungry data centers, andrelatively energy-efficient optical systems are seen as a potential way forward.
“Most of the interest in [photonics] is driven by the fact that AI inference has literally become limited by power, and in a rather striking manner,” says Peter McMahon, associate professor of applied and engineering physics at Cornell University, in Ithaca, N.Y., who has researched optical computing for over a decade.
Optics, at least on paper, carries the promise of doing the same math as today’s GPUs at a drastically lower level of power. Switching from electronics to optics would be a pretty radical solution. Nevertheless, like mushrooms after a rainstorm, optical-computing startups are popping up in the United States, China, Germany, and Israel. And while competing against such GPU makers as Nvidia is an ambitious goal, the investment in optical computing has spurred development of complementary approaches, such as designing alternative AI algorithms, taking on scientific computing, and enhancing in-network computing.
Optalysys, based in England, is inserting optical-computing elements into the networking path, where data centers already use light. The company’s basic on-chip optical element, shown in the inset, is a Fourier transform calculator.Optalysys
“I think it would be good for the technology if many of these companies made a good run at it,” says Logan Wright, assistant professor of applied physics at Yale University, in New Haven, Conn., and a former postdoc in McMahon’s lab. “I think that the technology that will be developed through this would be really, really useful, and enable photonic systems that maybe don’t get on this wave of AI but would be capable of doing really quite special things.”
Optics to inference’s rescue?
Inference makes up the bulk of AI energy use. The industry is already spending billions of dollars designing advanced GPUs and other chips to run inference as efficiently as possible. At the heart of the inference computation is one operation, done countless times a second: matrix-vector multiplication.
Back in 1978, optical-computing pioneer Joseph W. Goodman and his coworkers proposed a method for doing matrix-vector multiplication with light. The setup was relatively simple. Let’s say a vector of length N needs to be multiplied by a matrix of size N by N. N light-emitting diodes, all in a row, encode the elements of the vector into the intensity of light—the brighter the light from an individual diode, the greater the value that diode encodes. Then, a cylindrical lens spreads out the light vertically, essentially copying the vector N times. The light then passes through an attenuator of N by N pixels. At each pixel, the light is attenuated according to the corresponding value at that location in the matrix. Passing the light through the attenuator performs the multiplication. Finally, a cylindrical lens oriented perpendicular to the first lens re-collects the light from each row, summing it up. The final vector is measured by a row of detectors.
VECTOR-MATRIX MULTIPLICATION
The most computationally expensive part of common AI today is vector-matrix (or matrix-matrix) multiplication. For a vector of length N (here, N = 3), it takes N2 (here, 9) multiplication steps and three accumulation steps—multiple additions at once.
The argument for why optics might be advantageous over electronics goes something like this: Multiplying a matrix of size N by N by a vector of length N takes N2 multiplication steps. In electronics, each multiplication takes some energy, so the total energy used scales as N2. In optics, however, the multiplication step is passive: As long as the matrix stays the same, passing light through the attenuator costs almost no energy. The only cost is encoding the vector into light and reading out the answer, but the vector is of length N, not N2. So the energy used will scale as N, not N2, as bigger vectors and matrices are multiplied.
Since Goodman’s 1978 proposal, the quality of the available optical elements has improved dramatically. Indeed, in 2022, a team of researchers at Cornell led by McMahon demonstrated an optical neural network that consumed less than one photon of energy—approximately 2.5 x 10-19 joules—per multiplication operation. In a standard computer, a single multiply operation can take between 10-14 and 10-12 joules, which suggests that optics could possibly reduce energy use by a factor of 100,000.
But McMahon cautions against such extrapolation. First, the cost of encoding data into optics was not included in the calculation. Even if the scaling is favorable with light, in practice it’s not just the scaling that matters, it’s also the coefficients. Let’s say the energy cost of doing one multiplication with electronics is 1 picojoule, and the cost to encode one vector element with optics is 1,000 picojoules. The electronic approach still wins if the length N of the vector is fewer than 1,000 elements.
McMahon says this example is not far off from reality: “One of the key questions is, How big do you have to make the system before you start winning? Lots of people have done this calculation for various kinds of neural networks. The very rough answer is somewhere between N equals 1,000 and N equals 10,000.”
It’s also important to keep in mind a computing principle known as Amdahl’s law.
“Amdahl’s law shouldn’t even be called a law. It’s a simple piece of common sense,” McMahon says. “If you have something that’s going to accelerate a computation, and it can accelerate 90 percent of the code, but there’s 10 percent of the code that it can’t accelerate, then the maximum possible speedup you can get is 10x.”
With AI inferencing, the point is to save energy, not time. But Amdahl’s observation still applies. What fraction of today’s AI inferencing code can be better handled with optics is a complicated question that varies by approach. But it’s not 100 percent. Matrix-vector multiplication, or matrix-matrix multiplication, is only part of the inference workload—there are other, nonlinear operations that need to be performed on the data.
Taking all of these caveats into account, optical-computing companies today are aiming for a 10- to 80-fold improvement in energy efficiency over today’s GPUs. The question is whether they can deliver on that promise at scale before traditional electronics catches up to those gains.
Bridging the gap in free space
The startup Lumai, based in Oxford, England, has taken perhaps the most direct inspiration from Goodman and his team. Lumai’s head of product, Phillip Burr, declares that a free-space approach, where the light travels through air rather than on-chip waveguides, is what will enable the company’s tech to beat GPUs.
Instead of the fixed attenuator used in Goodman’s proposal, Lumai is using a modern programmable device called a spatial light modulator—a pixelated membrane akin to a fancy liquid-crystal display that can modulate both the intensity and phase of light. The rest of the design is more sophisticated in practice but similar in principle to Goodman’s design: A vector is encoded into light, copied N times via a lens, passed through the spatial light modulator, collected with another lens, and the result is converted back to digital electronics. The electronics then handle the nonlinear functions. “A full vector-matrix multiply completes in a single optical cycle,” Burr says.
To accurately compare the energy expenditures of the optical and GPU approaches, you must first specify which part of the workload you’re considering. Modern inference is often split into two phases: prefill, where the input query is processed and propagated through the model, and decode, where the output is generated one word (or token) at a time. Prefill is the more computationally intensive of the two. It basically involves doing vector-matrix multiplications for the same matrix over and over. Decode also involves vector-matrix multiplies, but not as intensively—the time-consuming part of decode is reading the necessary matrices in and out of memory.
Vector-matrix multiplication with free-space optics
The Goodman optical computer is the simplest version of a photonic vector-matrix multiplier. Here, the vector elements are encoded into the intensity of light from three light sources, such as LEDs. Then, a cylindrical lens spreads the light vertically. The matrix elements are encoded into the opacity of an attenuator, such that when light passes through each pixel it is multiplied (attenuated) by that matrix value. Finally, another cylindrical lens collects the light horizontally, performing the accumulation step, and a photodetector measures the output vector.
Lumai, and other companies in the space, are focusing on the compute-intensive prefill phase of inference. Lumai claims that its Iris Nova optical server, made available for evaluation in April 2026, is a proof of concept, and its next-generation Iris Aura server will do prefill using roughly one-tenth of the energy a GPU would use.
The startup Neurophos, based in Austin, Texas, is putting a different spin on the Goodman computer design. To pack more punch into a small device, the company is leveraging its expertise in metamaterials—complex structures that can manipulate light at scales smaller than a single wavelength. The structures it has designed are bigger than a wavelength but still quite small—just several micrometers per side.
“The equivalent of the optical transistor that you can get from standard silicon photonics today is like 2 millimeters long,” says Patrick Bowen, CEO of Neurophos. “You just can’t fit enough [of] them on-chip to get a compute density that remotely competes with the latest and greatest [that] TSMC [TaiwanSemiconductor Manufacturing Co.] does.”
Based in Austin, Texas, Neurophos is using tiny metamaterials packed onto a chip to manipulate incoming light for efficient vector-matrix multiplication.
Bowen’s team designed metamaterial-based modulators that can be programmed to manipulate the phase and intensity of light at the micrometer scale, resulting in an array of 1,000 by 1,000 modulators in a square chip with 5-mm sides. According to Bowen, Neurophos’s modulators switch over 10,000 times as fast as a standard liquid-crystal spatial light modulators do.
Bowen also claims a performance advantage over Nvidia’s Blackwell GPU: “We’re targeting 50 times Blackwell B200’s speed and 35 times its energy efficiency, and have a road map to reach 250 times the speed and roughly 90 times the energy efficiency in subsequent generations.”
The approach is CMOS-compatible, Bowen says, which should allow Neurophos to manufacture its chips on the semiconductor industry’s existing high-volume production lines. The company is currently choosing a site for a production facility ahead of its rack deliveries to customers in 2028.
Both Lumai and Neurophos are betting they’ll get their products to market before traditional CMOS can catch up to their energy savings. “Silicon will keep improving, but digital efficiency gains now come slowly and at rising cost,” Burr says. “Optics offers a different scaling relationship, in which efficiency improves as matrices grow. Larger models, longer context windows, and agentic workloads are all pushing prefill in that direction.”
Bridging the gap on-chip
Free-space optics carries the promise of almost energy-free multiplication, as in Goodman’s design. But another big advance in optics since Goodman’s work in the 1970s is integrated photonics, which allows you to guide and manipulate light on-chip.
“The name of the game is to do as large a vector size as possible to maximize the benefit of using optics, and for that reason the free-space approaches are promising,” says Yale’s Logan Wright. “But it’s still fundamentally harder” to use the free-space approach, “because there’s just so much less infrastructure than in silicon photonics.”
Other companies—like Shanghai-based Lightelligence and Q.ANT, based in Stuttgart, Germany—are trading the 3D advantage of free-space optics for the manufacturability of photonics integrated directly onto a silicon chip.
“There is a good reason that free-space optics are usually limited to 1,000 units a year,” says Q.ANT’s CEO and founder, Michael Förtsch. “The reason for this is lens inspection, assembly strategy, and so on. Semiconductor-manufacturable integrated photonic circuits are at this point way more promising to reach volume production.”
Vector-matrix multiplication with integrated photonics
One way to do vector-matrix multiplication with light on-chip involves encoding vector elements into the amplitudes of pulses of light, traveling sequentially through a waveguide. The multiplication is performed by Mach-Zehnder interferometers: A waveguide splits in two, and the phase of light traveling through one of the paths is shifted by a controlled voltage. When the two paths reunite, the resulting light intensity depends on that phase shift, performing a multiplication. Encoding the matrix elements into the phase shifts turns this into vector-matrix multiplication. A photodetector then collects the light from all three pulses, performing the accumulation step.
In Q.ANT’s approach, the input data is encoded into the phase and amplitude of light pulses via on-chip modulators. A vector in this scheme is represented as a sequence of such pulses, each representing one vector element. The light is guided around the chip with waveguides. The basic computational element is an interferometer: A waveguide splits into two paths, and the light beam traveling down one path is subjected to a phase shift, controlled by a set voltage. When the two paths recombine, the resulting interference performs a computation. Then, photodetectors and analog-to-digital converters turn the result back into digital data. The serialized vector-matrix multiplication approach is slower than Goodman’s computer, where the whole calculation happens at once. However, the slower speed can be mitigated by using multiple wavelengths at once, effectively encoding multiple vectors into light simultaneously.
Q.ANT’s Native Processing Server is already being used in research institutions like the Leibniz Supercomputing Centre in Munich and the Jülich Supercomputing Centre in Jülich, both in Germany. Earlier this year, the company announced a commercial sale to the cloud provider Ionos. Similarly, Lightelligence’s PACE2 processor is capable of multiplying 128-by-128-element matrices. In a paper released in 2025, the company described vector-matrix multiplication at a 1-gigahertz frequency and 3-nanosecond latency, claiming two orders of magnitude lower latency than with traditional GPUs.
The energy advantage is yet to be conclusively demonstrated, however, and Q.ANT’s Förtsch admits there is still a ways to go. The main issue is the energy cost of analog-to-digital conversion and back. As long as a fast, low-energy converter does not exist, Förtsch says, “we don’t see realistic chances that…a general vector-matrix multiplication machine in optics makes sense.”
Beyond matrix-vector multiplication
Not everyone in the field is focused on matrix-vector multiplication, despite it being all the rage in the AI inference world. In fact, while the optical vector-matrix multiply idea goes back to the 1970s, other computations with optics have been investigated as far back as the 1950s. For example, a Fourier transform—a common operation that decomposes a complex wave or signal into its constituent frequencies—can be done in optics with a single lens.
“A digital chip has to approximate [Fourier transforms] in many small steps, using hundreds to thousands of transistors and more compute cycles,” says Q.ANT’s Förtsch. “Models that use such nonlinear functions can reach the same or better accuracy with fewer parameters and fewer operations.”
Even though Q.ANT’s optical processor has been used for traditional vector-matrix multiplication, the company believes its greatest advantages lie in alternative AI architectures that leverage more complex functions. “Tasks such as image classification and segmentation rely heavily on operations like Fourier series, transformations, and other nonlinear functions, which photonics handles natively. We see similar potential in generative AI, especially image and video generation,” Förtsch says.
The Israel-based startup LightSolver is also targeting algorithms that are a more natural fit for its device. “If you look at propagating electromagnetic waves, essentially, they are running differential equations in their nature,” says Ruti Ben-Shlomi, CEO and cofounder of LightSolver. She says differential equations can be solved much faster and with higher energy efficiency on the company’s laser processing unit (LPU) than on a traditional computer.
LightSolver’s LPU is an interesting twist on the Goodman optical-computer design. It has input light, a programmable membrane in the form of a spatial light modulator, and output light, but the whole thing is placed in a cavity, where light bounces back and forth many times before reaching a steady state. In addition, the team places a gain medium inside the cavity. This gain medium functions much like those found inside a laser, causing the bouncing light to build up in intensity.
LightSolver, a startup based in Israel, is using free-space optics to quickly solve differential equations by harnessing the natural properties of light. LightSolver
Once a particular problem is encoded into the modulator, the gain medium is stimulated to emit random photons. They bounce back and forth, increasing in intensity when passing through the gain medium again, experiencing preprogrammed changes through the modulator and interacting with one another until they reach a steady state—the solution. Then, a simple camera takes a picture of the steady-state result. Using emulation, the LightSolver team recently showed that their LPU can solve differential equations 40 to 80,000 times as fast as a GPU, depending on the workflow. They are now working on explicitly benchmarking the energy improvements.
LightSolver’s team is currently working on AI applications for its LPU as well, using the company’s own algorithm to do image generation from text. “We’re not doing vector-matrix multiplication very fast. We are doing something different. We need to assume that the way you train the model, and you then do inference, is completely different. But if you do it differently, you’ll be able to do image generation with dramatic energy-consumption decrease and speedup,” Ben-Shlomi says. LightSolver has a working prototype of its LPU, and the team is working on a larger, rack-mountable box.
Yet another way to leverage light’s particular strengths is being pursued by Optalysys, a startup based in Leeds, England. Nick New, founder and chief product officer, has a background in free-space optics, and so he understood that taking a Fourier transform is very easy to do with light.
The key idea behind Optalysys’s technology is taking that Fourier capability of optics in free space and making it work on-chip. “Rather than using a lens these days, we have our own component, which is a passive component in the plane of the silicon that does that same job,” New says. And by cleverly designing that component with multiple outputs, the team figured out how to perform different operations as well. “Because of how the Fourier transform response is generated, you can actually pull out different mathematical operations from it,” New says. “We can produce very large, high-precision digital functions at superhigh efficiencies, using our analog photonics approach.”
“We’re effectively adding processing to where there is none right now.” —Nick New, Optalysis
Instead of redesigning an accelerator from scratch, Optalysys is targeting a piece of the AI workload that’s often already in the optical domain: networking. In the data center, racks of computers are connected to each other with fiber optics, and the optical connections are increasingly getting closer and closer to the GPUs and CPUs themselves.
Some computations already happen while the data is en route. These computations, which include combining results from multiple GPUs, are executed inside network switch cards. But the cards naturally use power to do that math. With passive optical components, these computations could be done essentially for free.
New says that in addition to communication between GPUs, Optalysys’s technology could be used to facilitate a GPU’s communication with memory, doing things like compression and format conversion on the fly. Going back to AI inference, this approach could improve the decode part of the workload. Unlike prefill, decode is memory bound: The most costly part is shuffling model data in and out of memory. “We’re more focused on the memory-bound applications, which fits very well with this principle of adding processing into the communication path,” New says. “We’re effectively adding processing to where there is none right now.”
The likely future of optical computing
How likely are optical computers to replace GPUs in the data center? Yale’s Wright is not too optimistic. “The direction of algorithm development is not going to be a direction that is automatically favorable for optics,” he says. “In fact, it’s most likely to go in the opposite direction, because [AI model developers] want it to work better on GPUs.”
However, even if our AI queries aren’t going to be computed in light in the next few years, that investment in the technology won’t be in vain. Says Wright, “The reason that I and many other people in the field are at least somewhat optimistic is that there’s real investment from companies like Nvidia and TSMC in developing co-packaged silicon photonics—not for computing, but for circuit switching and interconnects, and that just kind of opens the door to do a little bit more in the future.”
Whether that technology ends up crunching vector-matrix multiplication, doing bespoke simulations, or performing in-network computations, the computing ecosystem seems ripe for a variety of approaches. “We see the data center of the future as heterogeneous: CPUs, GPUs and photonic NPUs [neural processing units] each take on the tasks they are best suited for,” says Q.ANT’s Förtsch. “Running each task on the most efficient hardware can significantly reduce energy use and operating costs.”
{"imageShortcodeIds":[]}
原文为英文,正文由自动抓取并保留原文。
原文发布于 IEEE Spectrum AI(AI Is Giving Optical Computing Another Shot),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。