Meta Open-Weight Model Workbench

VRAM, Latency & TCO Analyzer for Open AI Models
Quick Presets:
Model & Hardware Setup
Model Parameters 70B
Quantization Precision FP8 (8-bit)
Target GPU Architecture
GPU Allocation Count 2 GPUs
Context Length (Tokens) 8,192
Concurrent User Streams 16
Avg Output Length (Tokens) 512
Hardware Requirement & Execution Telemetry Self-Hosted Simulation
Total VRAM Required
82.58 GB
Weights + KV Cache + CUDA Overhead
KV Cache Footprint
12.58 GB
16 streams @ 8k context
Min GPUs Required
2
NVIDIA H100 SXM 80GB
Time-Per-Output Token
14.20 ms
~70.4 tokens/sec/user
Cost per 1M Tokens
$0.42
Self-Hosted TCO estimate
VRAM Memory Allocation Breakdown (GB)
Latency Bottleneck (TTFT vs TPOT)
Self-Hosted TCO vs. Commercial Closed API Pricing
Deployment Route Hardware / API Cost / 1M Input Tokens Cost / 1M Output Tokens Estimated Blended Cost (1M)
Self-Hosted 2x NVIDIA H100 SXM $0.14 $0.70 $0.42
Closed API GPT-4o (Commercial API) $2.50 $10.00 $3.75
Closed API Claude 3.5 Sonnet $3.00 $15.00 $4.50
Generated Execution Flags (vLLM / TensorRT-LLM)
# Auto-generated vLLM production startup command python3 -m vllm.entrypoints.openai.api_server \ --model meta-llama/Meta-Llama-3.1-70B-Instruct \ --tensor-parallel-size 2 \ --quantization fp8 \ --max-model-len 8192 \ --gpu-memory-utilization 0.90 \ --max-num-seqs 16

Read the written memory terms before trusting the total

Read the explanation

The executable weight formula multiplies parameter billions by quantization bits divided by eight. At seventy billion, sixteen-bit weights give one hundred forty assigned gigabytes and eight-bit weights seventy. Bars use three pixels per assigned gigabyte. No metadata overhead factor appears in this term. This arithmetic assignment is separate from actual checkpoint storage, numerical quality and hardware allocation, none of which the page measures. The source cache approximation multiplies concurrent users, context plus requested output, an assigned zero point zero zero zero zero nine zero two eight coefficient, and parameter count relative to seventy billion. With sixteen users, eight thousand one hundred ninety-two context and five hundred twelve output, it rounds to twelve point five seven. Thirty-two users give twenty-five point one five. Bars use eighteen pixels per assigned gigabyte. The coefficient is a fixture approximation, not actual transformer KV geometry. The default displayed total adds seventy weight units and twelve point five seven cache units, giving eighty-two point five seven. Separately the chart includes a two point nine activation allowance, so its three slices sum to eighty-five point four seven. Bars use five point five pixels per assigned gigabyte. The executable total omits that displayed activation term. Preserve this difference when inspecting the source; the result is not proof of safe provisioning, current cost or measured latency.

Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.