LLM Architecture & KV Cache Memory Architect Meta Muse Glimmer 30B / Gemma Optimized

Model Architecture
Parameters (Billions) 30
Layers 48
Hidden Size 7168
Attention Heads 56
KV Heads (GQA) 8
Head Dimension 128
Context & Runtime Limits
Context Window (Tokens) 8192
Sliding Window (Tokens, 0=Disabled) 4096
Batch Size 4
VRAM & Roofline Diagnostics
Total VRAM Required
61.024
GB (Weights + KV + Overhead)
Model Weights VRAM
60.000
GB (BF16)
Total KV Cache Footprint
1.024
GB (Batch Size: 4)
KV Cache per Sequence
256.00
MB / sequence
Hardware Fit Analysis
NVIDIA H100 (80GB)
Fits
NVIDIA A100 (80GB)
Fits
NVIDIA RTX 4090 (24GB)
OOM Warning
Apple Mac Studio M3 Max (128GB)
Fits
Memory Bandwidth Decode Bandwidth (H100 - 3.35 TB/s)
Max Token Speed (Bandwidth-Bound)
124.50
Tokens / Sec (Single Stream)
Multi-Turn Conversation KV Cache Growth Simulator
Current Turns: 1
Model Weights VRAM: 60.000 GB
Total KV Cache VRAM: 1.024 GB
Total Required VRAM: 61.024 GB
H100 80GB Compatibility: FITS
RTX 4090 24GB Compatibility: OOM

Trace assigned weights and cache allocation units

Read the explanation

The written memory planner multiplies billions of model parameters by its assigned bytes per weight. An illustrative thirty-billion-parameter model at BF sixteen uses two bytes per weight and displays sixty GB; INT four uses one half and displays fifteen. Bars show those assigned weight figures at eight pixels per displayed GB. This excludes its zero assigned CUDA overhead and actual runtime workspaces, quantization metadata and allocator fragmentation; it is not a verified hardware fit. Its cache formula is two times layers times KV heads times head dimension times effective tokens times bytes per cache element. An illustrative thirty-two layers, eight KV heads, head dimension one hundred twenty-eight, one thousand twenty-four tokens, and two-byte elements produces one hundred twenty-eight MiB per sequence. Four simultaneous sequences produce five hundred twelve MiB. Bars show those cache values at zero point nine pixels per MiB. The source labels binary-divided cache totals as GB and adds them to decimal weight GB, so retain that unit limitation rather than certifying capacity. The effective token count is a positive sliding window only when it is smaller than the requested context. With the same illustrative cache dimensions, two thousand forty-eight effective tokens produce two hundred fifty-six MiB; a one-thousand-twenty-four window produces one hundred twenty-eight. Bars show equal units at two pixels per MiB. Hidden size and total attention head inputs do not enter this source cache formula. Its tokens-per-second label divides assigned bandwidth by model weight size and is not a measured decoding benchmark.

Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.