Why is an Older GPU Suddenly a Goldmine? The "RAMaggedon" Explained
In traditional 3D gaming, memory capacity beyond 8–10 GB rarely yielded performance increases. However, in local LLM inference (e.g., Llama 3, DeepSeek, Mistral, Qwen), VRAM capacity is a strict binary ceiling: if weights and KV-cache exceed GPU memory by even 100 megabytes, the workload falls back to system RAM over the PCIe bus, dropping generation rates from 80 tokens/sec to 1.5 tokens/sec. Older cards possessing 16GB, 22GB, or 24GB of high-speed memory have decoupled from normal silicon depreciation, surging on secondary markets.
VRAM Capacity vs Speed
Autoregressive decoding is memory-bandwidth bound. Each generated token requires reading every parameter in the model through the memory controller once.
KV-Cache Escalation
Longer context windows (8K to 128K tokens) store key-value attention vectors for previous tokens, demanding anywhere from 500MB to 16GB of additional dedicated VRAM.
Sell vs Keep Decision
Before offloading your RTX 3090 or modded card, assess whether buying equivalent modern 24GB VRAM (e.g. RTX 4090 at $1,800+) makes economic sense for your AI workload.