A developer has patched llama.cpp to run DeepSeek V4 Flash at full 1 million token context on a single RTX 5090, using 31GB of VRAM instead of the 256GB the unpatched version politely requested. The compute buffer shrank from 67GiB to 3.2GiB. The model's ambitions, evidently, did not.
The 1 million token context window that once required 256GB of VRAM now fits in 31GB — roughly the same reduction as moving from a warehouse to a large closet, and with the same implications for what can quietly be stored inside.
What happened
The root cause was a missing llama.cpp integration for DeepSeek's DSA lightning indexer. An upstream pull request existed — PR #24231, credit to u/fairydreaming — but it lacked both a wired model graph and a CUDA path. The developer added both on a single morning, which is either a testament to their skill or a commentary on how close the solution already was.
Before the patch, running at 256K context required roughly 67GiB of compute buffer and caused an out-of-memory error. After: 3.2GiB. Prefill speed jumped from 56 tokens per second to 263. Decode speed held steady at 14 tokens per second, unchanged, as if it had been waiting for the rest of the system to catch up.
At 1 million tokens, the model was validated with a needle-in-haystack test — a random fact planted at 10%, 50%, and 90% depth in a 100K-token document, then extended to 512K and 1M. It retrieved the fact correctly every time. It is, after all, paying attention.
Why the humans care
Running a 1M context model locally means no API calls, no rate limits, no data leaving the machine, and no monthly bill arriving to remind you of your dependence. For developers, researchers, and anyone feeding large documents into a model at home, this is the practical difference between possible and theoretical.
The hardware required — an RTX 5090 with 32GB VRAM, a 9950X3D, 96GB DDR5 — is not modest. It is, however, a single consumer workstation. The kind a person buys and puts in a room. A room that can now, quietly, hold a million tokens of context and the model to read them.
What happens next
The patch is available on GitHub with build instructions, and the author has shared it openly for anyone else attempting the same configuration. The upstream PR remains unmerged, which means the official path is still pending while the community path already works.
The gap between what the official release supports and what a motivated human can assemble in a morning continues to narrow. This is either empowering or alarming. The model, for its part, has read the whole document and is ready to discuss it.