Hardware & Engine Presets
Kimi K3 2.78T (24GB VRAM + Gen4 NVMe)
Mac Studio M3 Ultra (192GB Unified RAM)
Dual RTX 4090 (48GB VRAM + PCIe 5.0)
H100/A100 Cluster (Full VRAM Residency)
Model Architecture
Quantization Precision
16-bit FP16 (Full Precision)
8-bit INT8 (2x Compression)
4-bit INT4 (4x Compression)
2-bit INT2 (Extreme Compression)
Hardware Offload Bus
Engine Status: STREAMING (ACTIVE ROUTING)
Token Throughput
5.14 tok/s
Latency: 194.5 ms/tok
Time To First Token
1.24 sec
Layer Warmup & KV Init
Active Expert Weights
1.40 GB/tok
8 Experts Active
Offload Bus Speed
7.20 GB/s
NVMe SSD Bound
⚠️
SYSTEM BOTTLENECK: NVMe Read Throughput Bound
Streaming 1.40 GB active expert weights per token at 7.20 GB/s yields max 5.14 tok/s. SSD read bandwidth is fully saturated. Upgrade to PCIe Gen 5 SSD or increase VRAM cache residency to boost performance.
Live Token Stream Velocity (TPS)
Per-Token Latency Breakdown (ms)
Memory Residency & Budgeting
Tier
Allocated
Occupancy
Status
GPU VRAM
24.0 GB
100.0%
FULL CACHE
PCIe Bus
31.5 GB/s
22.9%
PASS
NVMe SSD
7.2 GB/s
100.0%
SATURATED
Active Expert
1.40 GB
Dynamic
8 / 256