EXL3 Trellis LLM VRAM Memory & Quantization Simulator

Architecture Config

Model Preset

Model Parameters 70.6 B

Context Window 4,096 tokens

Concurrent Batch Size 1

Quantization Scheme

Weight Precision Format

EXL3 Bitrate (BPW) 3.00 bpw

KV Cache Precision

Hidden Layer Count 80

Attention Heads (KV) 8 (GQA)

Total VRAM Required 0.00 GB Weights + KV + Overhead

Model Weights Size 0.00 GB Effective: 3.00 bpw

KV Cache Memory 0.00 GB 0.50 GB / sequence

Estimated Perplexity Delta +0.18 vs Native FP16 baseline

VRAM Allocation Breakdown & Trade-off Curve

Weights VRAM

KV Cache

Activations

Trellis Overhead

Calculated safety headroom margin included (1.0 GB baseline OS overhead)

Interactive codebook matrix node allocation density