• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.
  • The forums have been upgraded with support for dark mode. By default it will follow the setting on your system/browser. You may override it by scrolling to the end of the page and clicking the gears icon.

NVIDIA Commercializes Silicon Photonics with InfiniBand and Ethernet Switches

AleksandarK

News Editor
Staff member
Joined
Aug 19, 2017
Messages
3,201 (1.11/day)
NVIDIA has developed co-packaged optics (CPO) technology with TSMC for its upcoming Quantum-X InfiniBand and Spectrum-X Ethernet switches, integrating silicon photonics directly onto switch ASICs. The engineering approach reduces power consumption by 3.5x. It decreases signal loss from 22 dB to 4 dB compared to traditional pluggable optics, addressing critical power and connectivity limitations in large-scale GPU deployments, especially in 10,000+ GPU systems. The architecture incorporates continuous wave laser sources within the switch chassis, consuming 2 W per port, compared to the 10 W required by conventional externally modulated lasers in pluggable modules. This configuration, combined with integrated optical engines that use 7 W versus 20 W for traditional digital signal processors, reduces total optical interconnect power from approximately 72 MW to 21.6 MW in a 400,000 GPU data center scenario.

Specifications for the Quantum 3450-LD InfiniBand model include 144 ports running at 800 Gb/s, delivering 115 Tb/s of aggregate bandwidth using four Quantum-X CPO sockets in a liquid-cooled chassis. The Spectrum-X lineup features the SN6810 with 128 ports at 800 Gb/s (102.4 Tb/s) and the higher-density SN6800 providing 512 ports at 800 Gb/s for 409.6 Tb/s total throughput. The Quantum-X InfiniBand implementation uses a monolithic switch ASIC with six CPO modules supporting 36 ports at 800 Gb/s, while the Spectrum-X Ethernet design employs a multi-chip approach with a central packet processing engine surrounded by eight SerDes chiplets. Both architectures utilize 224 Gb/s signaling per lane with four lanes per port. NVIDIA's Quantum-X switches are scheduled for availability in H2 2025, with Spectrum-X models following in H2 2026.



Below is a video with more technical specs, in case you are interested, with nice showcase of what these switches are capable of.

View at TechPowerUp Main Site | Source
 
But i thought infiniband was a deprecated technology with everything going to ethernet now, with ultraethernet and whatnot
 
Back
Top