Model Architecture
Parameters (Billions)
30
Layers
48
Hidden Size
Attention Heads
56
KV Heads (GQA)
8
Head Dimension
128
Context & Runtime Limits
Context Window (Tokens)
8192
Sliding Window (Tokens, 0=Disabled)
4096
Batch Size
4
VRAM & Roofline Diagnostics
Total VRAM Required
61.024
GB (Weights + KV + Overhead)
Model Weights VRAM
60.000
GB (BF16)
Total KV Cache Footprint
1.024
GB (Batch Size: 4)
KV Cache per Sequence
256.00
MB / sequence
Hardware Fit Analysis
NVIDIA H100 (80GB)
Fits
NVIDIA A100 (80GB)
Fits
NVIDIA RTX 4090 (24GB)
OOM Warning
Apple Mac Studio M3 Max (128GB)
Fits
Memory Bandwidth Decode Bandwidth (H100 - 3.35 TB/s)
Max Token Speed (Bandwidth-Bound)
124.50
Tokens / Sec (Single Stream)
Multi-Turn Conversation KV Cache Growth Simulator
Current Turns: 1
Model Weights VRAM:
60.000 GB
Total KV Cache VRAM:
1.024 GB
Total Required VRAM:
61.024 GB
H100 80GB Compatibility:
FITS
RTX 4090 24GB Compatibility:
OOM