Wednesday, September 2nd 2026

Micron Explores Near-GPU NAND Flash to Run Bigger LLMs

Micron is reportedly exploring the idea of developing high-endurance NAND Flash modules that would be positioned closer to the GPU, rather than being located far away in the storage pool behind multiple protocols. The company is said to be investigating ways to bring NAND Flash closer to handle workloads that don't require the latency and bandwidth of traditional HBM or regular DRAM modules. This architecture, currently referred to as "near-GPU NAND," would serve as a middle ground between density and durability. This concept would allow GPUs to access a pool of NAND Flash that has a lower density than conventional TLC or QLC NAND, but instead focuses on improving I/O speeds, bandwidth, and read times.

For instance, this would enable a GPU to access a potential storage pool with hundreds of gigabytes of storage directly on the GPU package or PCB, similar to what DRAM like HBM or GDDR7/LPDDR6 already does. This tier would sit between GPU memory and regular system storage, acting as a very fast buffer, which would be ideal for the inference of massive LLMs. At that point, LLMs wouldn't be memory-bound and could run on systems with fewer GPUs, but with more storage and memory tiers to compensate for the required inference size. Additionally, this NAND would be significantly cheaper than HBM or regular DRAM and would be sized according to system specifications. Images below are HBF slides from SK hynix and Sandisk.
Micron is not alone in exploring this idea. For example, SK hynix and Sandisk have already standardized High-Bandwidth Flash (HBF) as a concept to extend the memory of a GPU's HBM space by bringing NAND closer and offering more durable NAND. Every company working on this idea would need to overcome the bandwidth and durability gap so that HBF or near-GPU NAND becomes a valuable complement to standard HBM memory.
Source: DigiTimes
Add your own comment

10 Comments on Micron Explores Near-GPU NAND Flash to Run Bigger LLMs

#1
Quitessa
Just effing Wonderful, more ways for them to screw over the average consumer with AI Only versions of cards that will be complete e-waste in a few years while jacking up the prices even more.
Posted on Reply
#2
ncrs
I'm expecting we'll end up with something like this (please excuse the crude MSPaint):

(source: Samsung's presentation at Hot Chips 2026)

With the HBM base die becoming a full-fledged chiplet, potentially with on-die compute, and manufactured on bleeding edge processes maybe HBF can be chained after HBM. On-die compute or the HBM controller could implement some magical caching scheme using HBF as bulk storage.
Posted on Reply
#3
BrainCruser
Put this on a CPU and it will be a godsend for databases. Forget AI, make a database specific CPU with this on top and mint gold.
Posted on Reply
#4
kazuviking
BrainCruserForget AI, make a database specific CPU with this on top and mint gold.
Dead within 5 years due to electron migration and current leakage.
Posted on Reply
#5
Buddyspike
QuitessaJust effing Wonderful, more ways for them to screw over the average consumer with AI Only versions of cards that will be complete e-waste in a few years while jacking up the prices even more.
I don’t see it this way , but rather like an opportunity to decrease the ridiculous demand on RAM. It’s ridiculous especially because with large MoE models it’s mostly used as read only storage for its weights. Of course There are other things like KV cache , buffers etc. but in theory , with enough bandwidth and reasonably low latency , access, things like non-shared /hidden expert weights could be stored on such NAND or perhaps Optane-like solutions instead.
Regarding specialized AI GPUs : we already have specialized data center and workstation GPUs tailored for AI , so I don’t think this would change much.
Posted on Reply
#7
igormp
This is a good idea as long as it can deliver enough bandwidth.
I wonder what the die area would be needed for both the device as well as the phy/controller needed for such thing.
Posted on Reply
#8
L'Eliminateur
BrainCruserPut this on a CPU and it will be a godsend for databases. Forget AI, make a database specific CPU with this on top and mint gold.
kazuvikingDead within 5 years due to electron migration and current leakage.
Exactly, nand has a fixed lifetime and even less if pushed this hard in a super high bw config close to a furnace(the main processor die spewing hundred of watts of heat right besides it). This is a product specifically designed for AI slop hallucination processors that will have a lifetime of months, maybe 2 years at the most and be thrown as ewaste.

You'd never fly a real CPU with a perishable low-lifetime failure-prone component like this on-chip, it's a warranty and part exchange nightmare waiting to happen, even less for a database server that needs the utmost reliability and lifetime or several years.
Posted on Reply
#9
Frank_100
This is a risky investment because Micron does not control the space on the GPU board.

Micron should use their profits from DRAM to build a ReRam foundry.

ReRam has the endurance. ReRam is heat tolerant.

That is the future.
Posted on Reply
#10
L'Eliminateur
Frank_100This is a risky investment because Micron does not control the space on the GPU board.

Micron should use their profits from DRAM to build a ReRam foundry.

ReRam has the endurance. ReRam is heat tolerant.

That is the future.
reram has extremely low density and speeds, it's not the future the same way 3d xpoint wasn't (even when it was light years ahead of nand flash in every metric save penny pinching)
Posted on Reply
Nov 19th, 2026 21:49 CST change timezone

New Forum Posts

Popular Reviews

TPU on YouTube

Controversial News Posts