Lexar's "AI Storage Stick" Concept Calls for Treating M.2 NVMe SSDs Like Memory-expansion Cartridges
If you think about it, an M.2 NVMe SSD isn't conceptually different from an NES cartridge, so you shouldn't need a USB/Thunderbolt enclosure to treat your M.2 SSD like a portable storage device, or so argues Lexar. The company innovated a concept for Mini-PCs and desktops that calls for a robust M.2 slot that can take many insertions, and for an M.2 SSD to be encased in a metal jacket (not an enclosure, just a jacket). The SSD is then inserted into a 25 mm-wide slot on the front-panel of a mini PC, directly into the M.2 slot that's wired to the processor or chipset, thereby eliminating all other overheads.
The Mini-PC you see is a prototype by ASUS. There is of course the problem that neither PCIe isn't practically a hot-pluggable standard, which is why Thunderbolt and USB 3 came to existence in the first place as hot-pluggable link layers on top of a serial physical layer. The company also showcased the AI-Grade Gen 5 x4 NVMe SSD. This M.2 NVMe drive that's probably based on a DRAMless controller. Driving Lexar's effort is the ability to treat the SSD like memory, so the LLM can load larger models, using a 3-tiered memory hierarchy. The GPU's video memory is at L1 due to its highest bandwidth and least latency to the GPU. Then comes the system memory, larger but relatively slower; and finally, NVMe SSDs. Lexar claims that this approach has found to relieve memory footprint by at least 40%.
The Mini-PC you see is a prototype by ASUS. There is of course the problem that neither PCIe isn't practically a hot-pluggable standard, which is why Thunderbolt and USB 3 came to existence in the first place as hot-pluggable link layers on top of a serial physical layer. The company also showcased the AI-Grade Gen 5 x4 NVMe SSD. This M.2 NVMe drive that's probably based on a DRAMless controller. Driving Lexar's effort is the ability to treat the SSD like memory, so the LLM can load larger models, using a 3-tiered memory hierarchy. The GPU's video memory is at L1 due to its highest bandwidth and least latency to the GPU. Then comes the system memory, larger but relatively slower; and finally, NVMe SSDs. Lexar claims that this approach has found to relieve memory footprint by at least 40%.
