Lightspeeding A.I.
ECE professor’s trailblazing startup shines light on energy-efficient AI infrastructure
by A.J. Kleber
It’s no secret that A.I. technologies, particularly generative A.I. technologies such as LLMs (Large Language Models), require enormous, and rapidly increasing, amounts of power to operate. The recently announced reopening of the infamous Three-Mile Island nuclear facility is just the latest item in the news cycle to bring the scale of this increase home to the public consciousness, even as the excitement and innovation surrounding all aspects of A.I. are at an all-time high.
The reason for this exploding energy consumption on the part of so-called Artificial Intelligence is the sheer quantity of data processing that is required, especially by popular generative AI-based applications such as ChatGPT, which are able to produce “original” written and visual works by learning patterns from immense datasets and using them to generate content to the user’s specifications. As A.I. technologies continue to develop and hit the market, their insatiable demand for more computing power is causing concerns about a “bottleneck” in data center hardware. Not only will energy demand continue to skyrocket, but so will the increased latency imposed by trying to move such an unfathomable amount of data through data centers which simply can’t handle the volume. As models grow, these processors will find themselves sitting idle an increasing fraction of the time, waiting for data exchange. In addition to the environmental cost of idle hardware burning precious energy resources, this kind of slowdown could “put the brakes” on innovation and the further development of these technologies and their potential economic benefits.
Our data infrastructure is on the brink of either a massive crunch, or a transformation that could alleviate these looming bottlenecks and even enable the energy-efficient scaling of computing hardware to support the growth of AI models. This is photonics pioneer and ECE faculty member Miloš Popović’s moment.
A vision for a brighter computational future
Professor Popović has been a leader in the development of silicon photonics for over fifteen years; pursuing the dream of replacing traditional electrical and copper-wire-based communications, the basis for all contemporary computing hardware, with cheaper, more energy-efficient, higher-bandwidth, further-reaching and above all, lower-latency light-based alternatives. He and his collaborators’ work led to the first microprocessor to communicate with other chips using light, originally demonstrated at CU Boulder, in 2015. That same year, he co-founded Ayar Labs, a company dedicated to translating that initial university-led research into practical and commercially viable technologies. These innovations have included the development of a commercial manufacturing platform for electronic-photonic integrated circuits in partnership with GlobalFoundries, a leading semiconductor manufacturer in the U.S., and the invention of the “optical I/O chiplet.” Now, that expertise is in high demand, because the integrated optical I/O technologies the company has developed based on Popović’s early research is poised to solve one of the computer industry’s most pressing problems.
Blazing a trail
In November 2023, President Biden and various luminaries had the opportunity to view a demonstration of a cutting-edge Intel/Ayar Labs collaboration at American Possibilities: A White House Demo Day, under the auspices of DARPA. The optical I/O chiplets on display allowed for extremely fast (4 Tbps each) chip-to-chip communication; fast enough for one chiplet to move the entire Library of Congress in minutes. That’s the kind of communication throughput required within data centers to support the growth of A.I., allowing for low–power, high-bandwidth networking between hundreds of GPUs within a disaggregated architecture that can make the most efficient use of hardware (and energy) resources.
The event was intended to showcase the results of federal investment towards future technological advancement, and it seems the “future” is now: according to Professor Popović, “the timeframe for optically-enabled GPUs going into production is well under 5 years from now.”
All eyes on optical I/O
While Ayar Labs is still “a couple years” ahead of the competition, according to Popović, they certainly aren’t alone in the belief that silicon photonics may hold the key to the data bottleneck problem, and beyond. Industry leaders from all over the globe are exploring the potential of in-package optical I/O chiplets, and Ayar Labs is busily partnering with companies like Intel, NVIDIA, and Lockheed Martin, as well as government agencies, to implement optical technology at scale.
Professor Popović’s revolution of illumination is well underway, prepared to challenge the limitations and electrical overconsumption of today’s A.I. technology in favor of a brighter future.
Professor Miloš Popović joined the Boston University Department of Electrical and Computer Engineering in 2016. His research centers on the development of cutting-edge photonic integrated circuit technologies for a variety of applications, including optical interconnects, but also quantum communication and computing, cryogenic signal processing, and lidar imaging. He is a Fellow of the National Academy of Inventors and of the David & Lucile Packard Foundation, has 41 issued and 15 pending patents. He was recognized for his mentorship with Supervisor of the Year Awards from both BU and the Northeast Association of Student Employment Administrators (NEASEA) in 2022.