How Optical Technology Will Save the Cloud

Introduction

The cloud computing industry is booming, with massive data centers springing up around the world to meet increasing demand. However, these facilities are facing growing pushback due to their heavy resource consumption, including straining local power grids, depleting water supplies, and even hindering housing development. As artificial intelligence (AI) workloads explode, requiring clusters with tens of thousands of GPUs and servers, the sustainability challenges will only escalate. A system with 32,000 GPUs could require 45 megawatts of power capacity - nearly 5 times the peak load of the Empire State Building.

Fortunately, optical technologies present a pathway to making data centers more efficient, sustainable, and better neighbors. Over the past 20 years, optical modules that convert electrical signals to optical signals for transmission over fiber optic cables have seen incredible improvements. Their data rates have increased 1000-fold while the power per bit has dropped by over 100 times.

Figure 1. Performance is up and power is down graph

These advancements in optical connectivity are key to enabling new, more distributed data center designs that can reduce the load on any single location.

Design Changes for the Data Center

While data center workloads grew 10X in the last decade, total power consumption remained flat thanks to efficiency efforts like allowing higher temperatures, improving cooling, and culling inactive servers. Optical fiber links now make it possible to further split up monolithic data center campuses into many smaller satellite facilities around 10,000 square feet in size. This allows the electrical load and other impacts to be spread across a region rather than concentrated.

Smaller data hubs could repurpose vacant commercial properties and use non-potable graywater for cooling. High-bandwidth 400Gbps and 800Gbps optical modules in very compact form factors provide the connectivity needed for this distributed model, reducing costs by 75% compared to traditional optics.

The 800Gbps ZR/ZR+ standard in particular overlaps well with bandwidth needs, as it can support full 800Gbps performance out to 500km - covering metro areas where over 75% of Europeans and North Americans live. These 800Gbps DSP-based modules double their capacity roughly every three years, with 1.6Tbps versions coming soon.

Figure 2. A silicon photonics device and its source wafer. By miniaturizing optical components and integrating them into the same piece of silicon, reliability, performance, and power can be simultaneously improved. Courtesy of Marvell Technology.

Additional Use Cases

While coherent DSP modules handle the inter-data center links, a different category called PAM4 DSP modules bridges the cabling within facilities from 5 meters out to 2km. These "active" optical or electrical cables with integrated DSP chips can boost throughput compared to conventional passive copper cables which become distance limited at higher data rates.

Looking ahead, optical links will start replacing copper cables altogether - first for emerging standards like Compute Express Link that allows pooling memory across chips, and eventually within racks and servers themselves. Though copper won't disappear overnight, its limits are being reached.

Breakthroughs in Core Optical Technology

While optical modules offer great efficiency gains, improving the core optical components themselves is also critical as the cumulative module power has grown 8X over 20 years despite the per-bit reductions.

Silicon photonics, which manufactures optical components like multiplexers and waveguides on conventional silicon chips, allows hundreds of components to be integrated into a single package. This lowers cost, power, and size compared to discretely packaging each optical element.

Silicon photonics is already used in the coherent DSPs for long-range ZR/ZR+ modules. Soon it will appear in the PAM4 modules for shorter data center links as well. Innovations like co-packaged optics, which integrate the optical components and driving electronics side-by-side, could reduce power for optical-copper connections by 20-50%.

While challenges remain, a roadmap is emerging for applying Moore's Law-like scaling to optics through Photonics Integration. Over the next 3-5 years, we should see these advanced packaging techniques and new architectures enabling further leaps in optical efficiency and performance.

Power consumption remains the biggest challenge facing the industry. But optical technologies, from smart interconnect strategies to integrated photonics, provide a clear path to doing more work with fewer resources. A combination of incremental improvements and breakthrough innovations in optics will be key to making the growing cloud computing infrastructure sustainable for the long run.

Reference

R. Nagarajan, "How Optical Technology Will Save the Cloud," Photonics Spectra, Mar. 2024. [Online]. Available: https://www.photonics.com/Articles/How_Optical_Technology_Will_Save_the_Cloud/a69706. [Accessed: 20, March, 2024].