The Evolution of Co-Packaged Optics for High-Speed Data Center Networks

Abstract

As data center networks scale to terabit speeds, new optical interconnect architectures like co-packaged optics (CPO) are emerging to provide the required bandwidth in an energy-efficient manner. This essay examines the motivation behind CPO, its implementation in the recently published CPO module standards from the Optical Internetworking Forum (OIF), and the potential challenges to adoption. While CPO promises significant power savings, questions remain around vendor lock-in, reliability, and continued pluggables advancement. Nonetheless, standards development is an important step towards CPO emerging as a disruptive optical technology.

Introduction

Data center networks are relentlessly scaling to keep pace with our data-hungry world driven by cloud, 5G, AI, and other bandwidth-hungry applications. As speeds ramp from 400G to 800G to 1.6T and beyond, existing electrical and optical interconnects are struggling to deliver the required bandwidth in an energy-efficient manner. This is motivating new optical interconnect architectures like co-packaged optics (CPO).

CPO involves tight integration of optical engines alongside switch ASICs to minimize interconnect loss and latency. This contrasts with the traditional model of using electrical traces to route signals to pluggable optics modules on the front panel. As this article explains, CPO can reduce the energy-per-bit by 5x or more compared to pluggables. However, realization of CPO comes with challenges around integration, thermal management, and vendor lock-in.

This essay examines the motivation for CPO, its implementation in new standards from the Optical Internetworking Forum (OIF), and the potential adoption challenges. Figures from the attached article are included to illustrate the CPO concept. While questions remain around CPO vs pluggables roadmaps, standards development is an important milestone for this potentially disruptive technology.

The Motivation for Co-Packaged Optics

The need for CPO is being driven by bandwidth and power constraints in data center networks, especially at the switch-optics interconnect. As data rates increase into the terabit realm, the energy inefficiencies of existing pluggable modules become untenable.

This is illustrated in Figure 1 from the attached article, showing the pluggable optics model where signals must traverse long PCB traces from the switch ASIC to front panel modules. This requires power-hungry SerDes driving up to 100W per switch for 400G Ethernet, according to Synopsys.

Figure 1: Evolving toward co-packaged optics. Source: Synopsys

CPO addresses this by integrating optical engines alongside switch ASICs on a silicon interposer or multi-chip module. This provides an ultra-short, low-loss interconnect that drastically reduces power. Per the article, CPO can cut energy-per-bit to below 3pJ, 5x lower than pluggables.

Besides power savings, CPO offers higher bandwidth density by integrating multiple parallel optics, along with lower latency from tighter integration. The proposed benefits make CPO an attractive option as data center networks scale up.

OIF Standards for Co-Packaged Optics

To enable interoperability of CPO modules across vendors, standards are required. The Optical Internetworking Forum (OIF) recently published implementation agreements covering 3.2Tb/s CPO modules.

The OIF standard in Figure 2 defines a CPO module with 8x 400Gb/s optical interfaces, supporting FR4 and DR4 modes. This provides up to 140Gb/s edge density within the CPO. The host interface is 32 x 112Gb/s extra short reach SerDes lanes compatible with 50Gb/s.

Figure 2: Evolving optics layouts — a slightly different view. Source: ASE

Key aspects of the OIF CPO standard include:

• High density 8x400Gb/s optical interfaces

• FR4 and DR4 modes for fiber vs lane configurations

• 32 x 112Gb/s extra short reach host electrical interfaces

• Backward compatibility with 56Gb/s host interfaces

• Support for optical engine and copper cable assemblies

• By defining interoperability specifications across data rate, reach, fiber modes, and host interfaces, the OIF standards enable multi-vendor CPO ecosystem development. This is an important step towards widespread CPO adoption.

Adoption Challenges for Co-Packaged Optics

While promising major benefits, CPO adoption faces some potential headwinds. Reliability, integration complexity, vendor lock-in, and continued pluggables advancement all pose challenges.

A key concern is failure modes and replacement. Since CPO modules integrate multiple high-speed links, any failure requires switch board replacement instead of just swapping a pluggable. The high power density also raises thermal management issues.

Vendor lock-in is another concern raised in the article. Hyperscalers prefer open pluggables markets and may resist CPO if it ties them to specific switch vendors. There are also integration challenges around aligning CPO development with switch ASIC roadmaps.

Finally, as the article notes, pluggables continue advancing with impressive cost and performance improvements. The industry may stick with pluggables longer than anticipated if they meet needs. However, in the long-term, CPO appears necessary to reach terabit speeds and beyond.

Conclusion

Co-packaged optics promises significant benefits for next-generation data center networks in terms of bandwidth density, power efficiency, and latency. The OIF’s CPO module standards enable an interoperable ecosystem. However, adoption faces challenges around integration, vendor lock-in, reliability, and continued pluggable advancement. If these can be overcome, CPO could emerge as a disruptive optical technology for datacom applications at 1.6Tb/s and beyond. The industry enthusiasm around CPO and progress on standards are positive signs for its future.