"Run Hermes for free" threads are everywhere. The claims are directionally right — open-weight models like Hermes 3 and Llama can cost radically less than premium APIs — but "free" always means someone else's subsidy, your hardware, or your time. This calculator compares four real setups at your usage level.
The four setups, honestly
Router APIs (OpenRouter-style)
One API key, hundreds of models. Open-weight models are dramatically cheaper than frontier ones: Llama-3.3-70B-class models commonly run around $0.10–$0.60 per million tokens versus ~$5–15/M input–output blended for premium closed models.
Watch for: provider variance in speed/quality, and ":free" variants that train on your prompts.
Free tiers
Genuinely $0 — with rate limits (often ~50–1,000 requests/day) and data-use trade-offs. Perfect for prototyping and hobby bots; unusable for production traffic. The subsidy can vanish any quarter.
Rule: never build a business on a tier you don't pay for.
Local inference (Ollama / llama.cpp / LM Studio)
A quantized 8B model runs on a 16GB laptop; a 70B needs ~48GB of VRAM/unified memory (e.g., dual 3090s or a high-RAM Mac). Marginal cost is electricity: a 350W GPU generating ~40 tokens/sec produces ~1M tokens in ~7 hours ≈ 2.5 kWh ≈ $0.42/M tokens at $0.17/kWh — but you paid for the hardware up front.
Privacy is total; capability tops out well below frontier models.
Rent-a-GPU (RunPod / Vast-style)
Rent an A100/4090 by the hour ($0.30–$2/hr) and serve the model yourself with vLLM. Beats per-token pricing once utilization is high — a $1.50/hr GPU pushing 1,500 tok/s serves ~5.4M tokens/hour ≈ $0.28/M — but idle hours bill anyway.
The break-even is a utilization problem, not a price problem.