Set Your Priorities
drag to rotate
—
—
The three building blocks
| Layer | What it is | Cost shape | Best at |
|---|---|---|---|
| Chat apps | Claude, ChatGPT, Gemini subscriptions ($20–200/mo) | Flat fee, generous limits | Interactive work: writing, analysis, coding sessions, agentic tools |
| Cloud APIs | Pay-per-token access to the same frontier models | Usage-based (e.g. $3–15 per million tokens) | Automation, pipelines, anything programmatic at scale |
| Local models | Open-weights models (7B–70B) on your own hardware via Ollama, LM Studio, llama.cpp | Hardware upfront, ~$0 marginal | Private data, offline work, unlimited high-volume tasks, fine-tuning experiments |
Rules of thumb the pros actually use
- Route by difficulty, not loyalty. A common pattern: local 8–30B model for classification, extraction, and drafts; frontier model for anything requiring multi-step reasoning. Difficulty routing can cut API spend 60–90%.
- The API break-even: at ~$5/M output tokens, a $2,000 local rig breaks even after roughly 400M generated tokens — that's heavy, sustained volume. Low-volume users should just use subscriptions.
- Privacy is binary. If data legally can't leave your machine (health records, client code under NDA), local isn't a preference — it's the only option. That single constraint often decides the whole stack.
- Model researchers use everything. People who build models (like the Hermes team the tweet asks about) typically prototype prompts in frontier chat apps, run bulk data generation through APIs, and eval on local checkpoints — three layers, one workflow.