MoE

Local MoE LLM VRAM & Memory Offload Planner

Hardware Bandwidth & Topology Simulator for Trillion-Parameter Models
Model Architecture
Total Parameters 2,800 B
Total Experts 64
Active Experts / Token 8
Quantization Precision
Context Window 131,072 tokens
Hardware Constraints
GPU VRAM Pool 96 GB
System RAM Pool 512 GB
PCIe Bus Bandwidth 64 GB/s
Estimated Throughput
12.4 tok/s
Bottleneck: PCIe Offload
GPU VRAM Usage
80 / 96 GB
Headroom: 16 GB
System RAM Usage
1336 GB
Requires Swap / Offload
KV Cache Footprint
16 GB
Allocated in VRAM
Memory & Bus Topology Map Topology Valid
VRAM Memory Allocation Breakdown 83.3% Used
Active Expert Weights (VRAM)
KV Cache (VRAM)
Free VRAM
Inactive Experts (System RAM)
Enjoy this tool? Build your own with Super