Wednesday, September 2nd 2020

NVIDIA RTX IO Detailed: GPU-assisted Storage Stack Here to Stay Until CPU Core-counts Rise

NVIDIA at its GeForce "Ampere" launch event announced the RTX IO technology. Storage is the weakest link in a modern computer, from a performance standpoint, and SSDs have had a transformational impact. With modern SSDs leveraging PCIe, consumer storage speeds are now bound to grow with each new PCIe generation doubling per-lane IO bandwidth. PCI-Express Gen 4 enables 64 Gbps bandwidth per direction on M.2 NVMe SSDs, AMD has already implemented it across its Ryzen desktop platform, Intel has it on its latest mobile platforms, and is expected to bring it to its desktop platform with "Rocket Lake." While more storage bandwidth is always welcome, the storage processing stack (the task of processing ones and zeroes to the physical layer), is still handled by the CPU. With rise in storage bandwidth, the IO load on the CPU rises proportionally, to a point where it can begin to impact performance. Microsoft sought to address this emerging challenge with the DirectStorage API, but NVIDIA wants to build on this.

According to tests by NVIDIA, reading uncompressed data from an SSD at 7 GB/s (typical max sequential read speeds of client-segment PCIe Gen 4 M.2 NVMe SSDs), requires the full utilization of two CPU cores. The OS typically spreads this workload across all available CPU cores/threads on a modern multi-core CPU. Things change dramatically when compressed data (such as game resources) are being read, in a gaming scenario, with a high number of IO requests. Modern AAA games have hundreds of thousands of individual resources crammed into compressed resource-pack files.
Although at a disk IO-level, ones and zeroes are still being moved at up to 7 GB/s, the de-compressed data stream at the CPU-level can be as high as 14 GB/s (best case compression). Add to this, each IO request comes with its own overhead - a set of instructions for the CPU to fetch x piece of resource from y file, and deliver to z buffer, along with instructions to de-compress or decrypt the resource. This could take an enormous amount of CPU muscle at a high IO throughput scale, and NVIDIA pegs the number of CPU cores required as high as 24. As we explained earlier, DirectStorage enables a path for devices to directly process the storage stack to access the resources they need. The API by Microsoft was originally developed for the Xbox Series X, but is making its debut on the PC platform.

NVIDIA RTX IO is a concentric outer layer of DirectStorage, which is optimized further for gaming, and NVIDIA's GPU architecture. RTX IO brings to the table GPU-accelerated lossless data decompression, which means data remains compressed and bunched up with fewer IO headers, as it's being moved from the disk to the GPU, leveraging DirectStorage. NVIDIA claims that this improves IO performance by a factor of 2. NVIDIA further claims that GeForce RTX GPUs, thanks to their high CUDA core counts, are capable of offloading "dozens" of CPU cores, driving decompression performance beyond even what compressed data loads PCIe Gen 4 SSDs can throw at them.

There is, however, a tiny wrinkle. Games need to be optimized for DirectStorage. Since the API has already been deployed on Xbox since the Xbox Series X, most AAA games for Xbox that have PC versions, already have some awareness of the tech, however, the PC versions will need to be patched to use the tech. Games will further need NVIDIA RTX IO awareness, and NVIDIA needs to add support on a per-game basis via GeForce driver updates. NVIDIA didn't detail which GPUs will support the tech, but given its wording, and the use of "RTX" in the branding of the feature, NVIDIA could release the feature to RTX 20-series "Turing" and RTX 30-series "Ampere." The GTX 16-series probably misses out as what NVIDIA hopes to accomplish with RTX IO is probably too heavy on the 16-series, and this may have purely been a performance-impact based decision for NVIDIA.
Add your own comment

52 Comments on NVIDIA RTX IO Detailed: GPU-assisted Storage Stack Here to Stay Until CPU Core-counts Rise

#51
Mouth of Sauron
ValantarAll I'm saying is that you're making some claims here that you're not backing up with anything of substance, beyond alluding to experience as if that explains anything. I'm not doubting your experience, nor the value of it - not whatsoever - but all that tells us is that you ought to know a lot about this, not what you know. Because that is what I'm asking for here: an explanation of what you are saying. I'm asking you to present your points. I'm not contesting your claims (well, you could say that about decompression and RT audio, but you don't seem interested in discussing those), but you said something to the effect of current RT being complete garbage, which... well, needs fleshing out. How? Why? On what level? I mean, sure, we've all seen the examples of terrible reflection resolution in BF1 etc., but it seems you are claiming quite a bit more than that - though again, it's hard to judge going by your vague wording. So maybe try not to be insulted when someone asks you to flesh out your claims, and instead ... do so? Share some of that knowledge that you - rather patronizingly, I might add - claim that I should accept on blind faith? I think we're both talking past each other quite a bit here, but as far as I understand the general attitude on any discussion forum, making a vague claim - especially one backed by another claim of expert knowledge - and then refusing to go beyond this vagueness is a rather impolite thing to do. You're welcome to disagree, and I'll be happy to leave it at that, but the ball is firmly in your court.

I was also interested in seeing you argue your points about both the two first points in the post you quoted (about GPU-accelerated decompression and "RT" audio), but again you don't seem to have come here to have any kind of exchange of opinions or knowledge. Which is really too bad.


I would assume on Nvidia's internal servers and the work computers of their employees, and nowhere else. Nvidia doesn't tend to be very open source-oriented.
OK, perhaps I have misunderstood your meaning - what it looked like to me is that you said that "I need to flesh out stuff for myself (and perhaps return then)". This doesn't sound nice at all, you'll admit. Again, I'm not (surprise!) native speaker, so I may've got it wrong - the whole thing sounds quite different with that premise...

RT is not the topic here, but I'll be brief - no, I don't expect people to believe me blindly. Then again, none of the new products is out and tested yet, so discussing them now is kinda pointless - this is inline my post that I (personally) have low confidence in NVIDIA claims before it (testing) proves them.

What is certain is that they purposely use the terms wrongly, in marketing purposes. From computational point of view, real camera or scene RT is still very far away, also across the whole range of hardware we will have rasterization + RT combination, done by different shaders. All that for, according to Steam, market that currently has significantly less than 5% 'RTX' GPUs. Old games won't be 'upgraded' in 99% cases. Those are the facts.

How game devs will react and what will be adoption rate, also how their interpretations (it's not all about hardware) across a hardware line of different capabilities will look like is questionable. RT is so uneven in computational load that FPS is extreme sawtooth - this applies the most for full scene/camera RT, but also partial interpretations (just lights and reflections, just shadows etc). To compensate this, they will use a lot of approximations - and don't hide it (this may not be necessarily a bad thing, could be an achievement, really - except, well, it's untested too). Those are the questions.

At the end - and this got quite longer than intended - my personal reason to have a low confidence in NVIDIA is that they used terms improperly for marketing, claimed that various partial solutions and approximations are 'the holy grail' of rendering (RT itself, by the way, isn't perfect for that matter), which will provide highly impactful results. Well, I need to see that firstly - and they are obviously in overhype-mode right now, claiming miracles everywhere.
Posted on Reply
#52
InVasMani
PunkenjoyThe compression is not just useful for saving SSD space but also for bandwidth saving.

Let say you have a link that can send 10 GB/s. You want to send 2 GB uncompress or 1 GB compress. The first one will take at least 200 ms where the second one would take 100 ms.

This is just for pure data transfer but you can see how it can reduce latency on large transfers

Also, these days the major energy cost come from moving the data around and not from doing the calculation itself. If you can move the data in a compressed state, you can save power there.

But what I would like to know is can we just uncompress just before using it and continue to save on bandwidth and storage while it sit in GPU memory? Just in time decompression!

That do not seem to do that there but I think it would be the thing to do as soon as we can have decompression engine fast enough to handle the load.




I think the future might be interesting. If AMD want to be the leader on PC, they might bring to PC OMI (Open memory interface) where the memory or storage is attached to the CPU via a super fast serial bus (Using way less pin and die space than modern memory technology). The actual memory controller would be shifted directly on the memory stick. The CPU would become memory agnostics. You could upgrade your CPU or memory independly. Storage (like optane) could also be attach via this.

The pin count is much smaller than with modern memory so you can have way more channel if required.


This is based on the OpenCAPI protocol. OpenCAPI itself would be used to attach any kind of accelerator. The chiplet architecture from AMD would probably make it easy for them to switch to these kinds or architecture and it's probably the future.

These are open standard pushed by IBM but i would see AMD using them or pushing their own standard in the future that have a similar goal. With these standard, the GPU could connect directly to the Memory controler and vice versa.
VRAM can be compressed NTFS compression or other more advanced compression methods like LZX. I'm not sure how that works with data actually being utilized though, but a lot of data is allocated into VRAM that isn't actually being used at all times it's literally sitting there so when it needs to be accessed it's ready to be and can quickly. That of course begs the question if it could be decompressed on the fly quickly and when it's not readily being utilized re-compressed to fit more data into the VRAM as well increase bandwidth since compressed data that can be decompressed quickly by the CPU can actually end up being faster. What you say about the compression saving on bandwidth is defiantly true I can actually get more bandwidth with compressed storage on a VRAM disk or a RAM disk with NTFS compression enabled in certain instances. In fact I disabled a CPU core and thread on my CPU and tested it and performance dropped under the same settings by a noticeable amount pretty much across the board within the same test scenario which seems to indicate the CPU core count and cache per core comes into play in a noticeable way. In fact over the years with gaming we've seen a number of times over CPU cache come into play so I'm not surprised in the least.
Posted on Reply
Add your own comment
Nov 18th, 2026 23:26 PST change timezone

New Forum Posts

Popular Reviews

TPU on YouTube

Controversial News Posts