Why VRAM Matters

For running AI locally, a powerful GPU like the GeForce RTX 5090 is a superb and obvious option (if you have the money). Whether the workload is a large language model, an image generator or anything else, the fastest consumer graphics card money can buy will chew through it faster than this little AMD box, on one condition: the model has to fit inside the 5090's 32 GB of video memory. That one condition is the whole story of the Ryzen AI Halo, and we measured it.

I ran several models of various sizes, ranging from half a billion parameters to 235 billion, and measured how fast each one generates text. The result is plotted as "how much faster" the Ryzen AI Halo is compared to the RTX 5090. Below the 1x line, the 5090 wins; above it, the Halo does. Read it from left to right.


For every model small enough to fit inside the RTX 5090's 32 GB of memory, up to roughly 30 billion parameters, the 5090 is the faster machine by a big margin, as expected. The 5090 is a far more powerful GPU in terms of number crunching performance. What I found interesting is that the gap widens with bigger models.

Then the model stops fitting, and the RTX 5090 falls apart. A 72-billion-parameter model needs about 41 GB, a bit more than the 5090's 32 GB, so the card has to push the excess into system RAM over the slow PCIe bus. With an even bigger model, 235 billion parameters, the Ryzen AI Halo crushes the RTX 5090 and is almost 20 times (!) faster, because it can hold the entire model in its 128 GB of unified memory.

Serious high-end "frontier" models leave 32 GB behind almost immediately. Quantized to 4-bit, the usual way to run them locally, a model costs roughly half a GB of memory per billion parameters, so even the smallest thing anyone calls frontier, a 70-billion-parameter model, already needs about 40 GB, more than the 5090 can hold. The highly popular newer open models actually go even bigger: gpt-oss-120b wants around 64 GB, Qwen3-235B about 85 GB even squeezed down to 2-bit, and the largest open releases, Llama's 405B and DeepSeek-V3 at 671 billion parameters, land somewhere between 230 and 400 GB before a single token of context. And context is not free either; a long conversation's key-value cache can add tens of gigabytes on top.

In a datacenter, like when you're talking to Claude or ChatGPT, the models run at higher precision, so we're talking about 1 TB of RAM and more spread across multiple GPUs. A common setup is an eight-GPU server (8x H100 or H200 accelerators, 640 GB to 1.1 TB of HBM between them), that costs several hundred thousand dollars and draws multiple kilowatts of power.

The RTX 5090's 32 GB is of course not the end of the VRAM sizes, but you'll certainly have to pay up if you're considering those RTX Pro cards. NVIDIA's data-center cards hold these models with room to spare: an H100 carries 80 GB, an H200 141 GB, and they run them far faster than anything here. They also cost on the order of $25,000 to $40,000 each, need a server to house them, and draw many times the power. Blackwell is even more expensive. Memory is the expensive part of AI hardware nowadays, which is also the reason why AMD's 128 GB box costs $4,000.