Local AI Agent VRAM Profiler llama.cpp & Ollama

Interactive layer offloading memory profile & token bandwidth simulator

✓ Model fits completely in GPU VRAM! Maximum generation speed achieved.
Offloaded Layers
32 / 32
100% on GPU VRAM
Total VRAM Allocation
5.82 GB
48.5% of 12.0 GB VRAM
KV Cache Footprint
0.50 GB
8,192 tokens (GQA Enabled)
Est. Token Speed
86.6 t/s
Bandwidth bottlenecked
Visual Layer Stack & Memory Partitioning Real-time D3.js Allocator
VRAM Model Layers
KV Cache Allocation
CUDA Overhead / Context Buffer
System RAM Offload
OOM Memory Overflow
./llama-cli -m models/llama-3-8b.Q4_K_M.gguf -ngl 32 -c 8192 -b 1 --numa distribute
Runtime Evidence Vector: Llama 3 8B @ Q4_K_M with 8192 context fits 32/32 layers in 12.0GB VRAM. Estimated speed: 86.6 tokens/sec.

Trace the saved memory allocations and calibration branches

Read the explanation

For the default eight billion parameters, the source assigns four point five bits per Q4 K M weight. Dividing by eight gives four point five decimal gigabytes; twelve percent tensor allowance raises this to five point zero four. Both bars use ninety pixels per decimal gigabyte. The source quantization lookup includes its own format overhead, and the extra factor is a planning allowance. Neither the label nor this estimate proves numerical quality or actual allocator usage. The written cache equation derives a hidden dimension from the square root of eight billion parameters times one thousand twenty-four. With thirty-two layers, batch one and eight thousand one hundred ninety-two context tokens, it gives about point three five three five five three gibibytes. Doubling context doubles that result. Both bars use six hundred pixels per gibibyte. This inferred architecture and fixed GQA division are heuristics, not a measured model cache shape. The runtime allowance is point six plus point three when context is at most thirty-two thousand seven hundred sixty-eight. Above that threshold it becomes point six plus point eight. The totals are point nine and one point four, using three hundred pixels per assigned gigabyte. The layer-fit rule then subtracts cache and runtime from capacity and floors the remaining space divided by weight per layer. Its weighted-bandwidth token rate is a heuristic, not a throughput benchmark.

Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.