This Light-Powered AI Chip Is 100x Faster Than a Top Nvidia GPU

As generative AI models grow more powerful, their energy use is becoming a serious bottleneck. A new fully optical generative AI chip could help by running advanced image and video generation tasks at speeds and efficiencies orders of magnitude beyond today’s hardware.

Training generative AI models requires an enormous amount of computing power and energy. But as demand explodes, the process of actually running the models to create images, text, or video—known as inference—is quickly becoming an even bigger drain on resources.

Video and image generation models are particularly energy intensive. While the efficiency of these models is constantly improving, a 2023 study found that generating 1,000 images using a leading model produced carbon emissions equivalent to driving a gas-powered car more than four miles.

One promising approach for slashing energy use is photonic computing, where processors use light instead of electricity. It’s a tactic multiple well-funded startups are pursuing in earnest. But most advances have been limited to simpler tasks like image classification or text generation.

Now, researchers from Shanghai Jiao Tong University and Tsinghua University in China have demonstrated an all-optical chip they call LightGen that is more than 100 times faster and more energy efficient than a leading Nvidia GPU on tasks like video and image generation.

“LightGen provides a new way to bridge the new chip architectures to daily complicated AI without impairment of…