Nov 72 min read

Marvell's AI Accelerator: Powering the Future of Hyper-Scale Computing

Introduction

In the rapidly evolving world of data center infrastructure, Marvell has emerged as a key player in the AI accelerator market. At their recent AI-focused financial analyst day, the company unveiled a series of exciting announcements and insights that shed light on the future of this dynamic industry.

Marvell's Custom Compute Offerings

One of the standout announcements from the event was Marvell's new AI accelerator win at a third US hyper-scaler. This adds to the company's growing portfolio of AI inference accelerators and custom Arm CPUs, reflecting its strategic focus on collaborating with major tech giants to develop tailored solutions.

Marvell's approach to custom compute is centered on leveraging its extensive IP portfolio to build multi-reticle-size chips into a single package. This flexibility allows the company to work closely with customers to create specialized solutions that cater to their unique needs.

Figure 2. Marvell Custom Compute AI Accelerator Q2 2024

Accelerated Compute Market Opportunity

Marvell's analysis of the data center infrastructure market paints a compelling picture of the growing demand for accelerated computing solutions. While traditional general-purpose compute is expected to see a modest 3% CAGR, the market for accelerated compute is projected to experience a remarkable 32% CAGR.

Figure 3. 2023 Data Center Infrastructure Market Marvell

Within this accelerated compute market, Marvell sees an even faster growth trajectory for the custom compute segment, with a forecasted 45% CAGR. This trend is largely driven by the increasing preference of hyper-scalers to build their own custom chips.

Figure 4. Marvell Q2 2024 DC Custom Compute Opportunity

Interconnect Advancements

Interconnects play a crucial role in the AI infrastructure ecosystem, and Marvell is at the forefront of this space. The company's investment in switching R&D has increased by 2.5x, with the Innovium-derived Teralynx 10 51.2T product starting to ramp up.

Figure 5. Marvell 2024 Data Center Switching RD Investment And Teralynx

Marvell also highlighted the growing importance of interconnect speeds, with the industry transitioning to a faster cadence of generational shifts. This accelerated pace is driven by the need to decrease the cost per bit of data movement and enhance overall performance.

Figure 6. Optical Interconnects Growing Faster than XPUs

Figure 7. Marvell Q2 2024 AI Interconnect Speed Increases

Marvell's 3D SiPho Engine

One of the most intriguing product announcements from the event was Marvell's 3D SiPho (Silicon Photonics) engine. This technology integrates 32x 200G links on a single chip, offering a versatile solution for various applications.

Marvell's customers may not be ready for the 51.2T co-packaged optics switches discussed by Broadcom, but the company has made significant strides in developing SiPho technology that can scale cost-effectively as interconnect speeds increase.

Conclusion

Marvell's AI Investor Day 2024 showcased the company's strategic focus on the accelerated compute market, with a particular emphasis on custom solutions for hyper-scalers. The insights shared on the growing market opportunity, advancements in interconnect technology, and the innovative 3D SiPho engine highlight Marvell's positioning as a key player in shaping the future of AI infrastructure.

As the data center landscape continues to evolve, Marvell's commitment to innovation and collaboration with industry leaders will undoubtedly play a pivotal role in driving the next wave of hyper-scale computing.

Reference

[2] P. Kennedy, "Marvell AI Investor Day 2024 New US Hyper-Scale AI Accelerator," ServeTheHome, 12-Apr-2024. [Online]. Available: https://www.servethehome.com/marvell-ai-investor-day-2024-mrvl/. [Accessed: 13-Apr-2024].