NVIDIA Prepares GB200 NVL4: Four "Blackwell" GPUs and Two "Grace" CPUs in a 5,400 W Server

[AleksandarK]

At SC24, NVIDIA announced its latest compute-dense AI accelerators in the form of GB200 NVL4, a single-server solution that expands the company's "Blackwell" series portfolio. The new platform features an impressive combination of four "Blackwell" GPUs and two "Grace" CPUs on a single board. The GB200 NVL4 boasts remarkable specifications for a single-server system, including 768 GB of HBM3E memory across its four Blackwell GPUs, delivering a combined memory bandwidth of 32 TB/s. The system's two Grace CPUs have 960 GB of LPDDR5X memory, making it a powerhouse for demanding AI workloads. A key feature of the NVL4 design is its NVLink interconnect technology, which enables communication between all processors on the board. This integration is important for maintaining optimal performance across the system's multiple processing units, especially during large training runs or inferencing a multi-trillion parameter model.

Performance comparisons with previous generations show significant improvements, with NVIDIA claiming the GB200 GPUs deliver 2.2x faster overall performance and 1.8x quicker training capabilities compared to their GH200 NVL4 predecessor. The system's power consumption reaches 5,400 watts, which effectively doubles the 2,700-watt requirement of the GB200 NVL2 model, its smaller sibling that features two GPUs instead of four. NVIDIA is working closely with OEM partners to bring various Blackwell solutions to market, including the DGX B200, GB200 Grace Blackwell Superchip, GB200 Grace Blackwell NVL2, GB200 Grace Blackwell NVL4, and GB200 Grace Blackwell NVL72. Fitting 5,400 W of TDP in a single server will require liquid cooling for optimal performance, and the GB200 NVL4 is expected to go inside server racks for hyperscaler customers, which usually have a custom liquid cooling systems inside their data centers.



View at TechPowerUp Main Site | Source

 

[windwhirl]

Maybe it's just me, but isn't 5400 W a bit high of a TDP for the amount of GPUs and CPUs included?

 

[Daven]

Maybe it's just me, but isn't 5400 W a bit high of a TDP for the amount of GPUs and CPUs included?

I'm guessing 300W per CPU and 1200W per GPU. Yes that's quite high for the GPUs and unsustainable in my book. On this trajectory, Rubin with be a 2000+ W GPU!!!

 

[bonehead123]

And folks wonder what "Global Warming" is or what's causing it.....

wait for it........wait for it........

Tah Dah...

It's ALL nGreediya's fault, hahahahaha

 

[igormp]

I'm guessing 300W per CPU and 1200W per GPU. Yes that's quite high for the GPUs and unsustainable in my book. On this trajectory, Rubin with be a 2000+ W GPU!!!

The GB200 supposedly has a TDP of 1000W and each grace CPU is ~250W with memory, but then you have the huge interconnects, IO and all the conversion for that huge chip.

 

[windwhirl]

Ok, so it's just Nvidia letting the chips run wild, pretty much.

 

[abysal]

5.400 Jiggawatts! Great Scott! Oh common, this is nothing some bought carbon credits won't solve. Meanwhile 15 min cities selling carbon credits after making it illegal for you to drive an ICE vehicle.