Tech BusinessChips & Semiconductors
High-Bandwidth Flash research targets GPU memory latency
A research paper from Georgia Tech, NVIDIA Research and Stanford proposes SlipStream, a method to reduce latency and buffer overhead when using flash-based memory in GPU systems.

Key takeaways
- Georgia Tech, NVIDIA Research and Stanford researchers published a SlipStream paper on High-Bandwidth Flash for GPU systems.
- HBF can offer up to 16 times more capacity than HBM, but read latency can be about 20 times higher.
- SlipStream uses demand-paced staging and 4 KB page tracking to reduce buffer needs.
- The paper reports 92.4% of ideal HBF performance with a 40 MB buffer, versus 43.1% for H3 with 320 MB.
What the research targets

According to Semiconductor Engineering, researchers at Georgia Tech, NVIDIA Research and Stanford University published a study. It examines High-Bandwidth Flash, or HBF, for GPU systems. The work is dated 16 November 2026. It focuses on a method called SlipStream. The method aims to make flash-based memory more usable for large language model inference.
The paper says large language model inference needs more memory capacity. This is because models and batch sizes are growing. HBF is presented as a possible answer. HBM is the high-bandwidth memory technology used in the systems studied. HBF swaps its DRAM layers for NAND flash. NAND flash is the memory type used instead of DRAM in HBF. The reported benefit is up to 16 times more capacity. Read bandwidth remains comparable.
The trade-off is delay. The paper reports read latency of nearly 20 microseconds. That is about 20 times the latency of HBM. When HBF is accessed on demand, inference can slow by up to 12.8 times. The paper also addresses SRAM overhead. This is a cost linked to fast memory used for tracking.
How SlipStream works
The paper studies prefetching. This means data is fetched ahead of demand. It says conventional cache-line prefetchers would need 20 times more tracking state. That extra state would cover requests in flight and hide the delay. Another class of designs, including one called H3, follows virtual-address order, the order of memory addresses. It does not follow the order required by GPU kernels. The paper says such designs need hundreds of megabytes of buffering.
SlipStream combines software and hardware changes. In software, it profiles the repeatable order in which weight pages are demanded. Weight pages hold model weights. This applies to data-independent LLM kernels, or operations with repeatable access patterns. It then inserts a staging warp. This is a GPU thread group that stages data. The warp replays the order using STAGE instructions that do not block. In hardware, the staging warp produces data. Work warps consume it.
A key idea is to remove each STAGE tracking entry early. The entry is removed when the demand side first touches the page. It is not removed when the page returns from HBF. This avoids keeping tracking entries alive until the flash read completes. The design tracks 4 KB pages rather than cache lines. Buffer slots are assigned only after pages return. This separates buffer size from traffic still in flight.
Reported performance
Across seven dense models spanning 16 GB to 800 GB, the paper reports results. SlipStream reached 92.4% of the performance of an ideal HBF. That ideal HBF would have HBM-like latency. The test used a 40 MB staging buffer. H3 reached only 43.1%. It reached that level even when given a 320 MB buffer.
The paper also says SlipStream buffer sizing does not depend on HBF latency or workload. A 16 MB buffer performed within 4% of the 40 MB version. That buffer is 20 times smaller than the one used by H3. These results are the paper's reported figures. They are not a commercial product claim.
Why it matters
The work matters to designers of GPUs, AI accelerators and memory systems. Large language models need more memory as they grow. If flash can provide more capacity without high delay, systems could use less DRAM for some workloads. The paper's approach tries to keep data moving ahead of demand. It also tries to avoid large buffers.
The paper is by Anish Saxena, Hritvik Taneja, Christos Kozyrakis and Aamer Jaleel. It appears as arXiv preprint arXiv:2611.14776. The material describes a research method, not a new memory chip or commercial product. It targets latency and buffer overhead in GPU systems. The claims come from the paper as summarised by Semiconductor Engineering. The performance figures are reported research results.
Sources
- Reducing High-Bandwidth Flash Latency In GPU Systems (Georgia Tech, Nvidia, Stanford)Semiconductor Engineering
Spotted a mistake? See our corrections policy. Read how News by OnDevice works in our editorial policy.



