Where does your prompt actually go?
Green orbs are your prompts. In local mode they orbit your house; in cloud mode they stream across the internet to a data center — faster models, but every token transits third-party hardware. Drag to rotate.
The real trade-off table
| Factor | Local (e.g. 8–70B open model) | Cloud API (frontier model) |
|---|---|---|
| Privacy | Prompts never leave your LAN | Provider ToS + retention policies apply |
| Quality | Good for drafting, summaries, RAG over your files | State of the art reasoning & coding |
| Up-front cost | $600–$2,500 (used RTX 3090/4090 or Mac w/ 32–64GB unified memory) | $0 |
| Marginal cost | Electricity: ~0.35 kW × $0.15/kWh ≈ $0.05/hr under load | Roughly $0.15–$15 per million tokens depending on model |
| Speed | 15–60 tokens/sec typical on consumer GPUs | 50–200+ tokens/sec, plus 50–300 ms network RTT |
| Uptime & ops | Yours to maintain (updates, cooling, remote access) | Provider's problem |
Worked example: the break-even
Say you burn 1M tokens/day on mid-tier cloud models at a blended ~$1.50 per million tokens: $45/month. A used RTX 3090 rig costs ~$1,100 and about $11/month in power at 2 hrs/day load. Break-even: $1,100 ÷ ($45 − $11) ≈ 32 months — and you got a lower-quality model. The honest conclusion: self-hosting rarely wins on pure cost. It wins on privacy (medical notes, client data, contracts), on unlimited experimentation without a meter running, and on offline resilience. That's why the sensible pattern is hybrid: local model for sensitive/bulk work, cloud API when you need frontier reasoning.
If you want to try it this weekend
1) Install Ollama or LM Studio. 2) Pull an 8B open-weight model (Llama or Qwen class) — it runs on 16 GB RAM. 3) Point any OpenAI-compatible client at http://localhost:11434. 4) For phone control, add Tailscale so your devices share a private network — the same "control from anywhere, data stays home" trick the SoloHost pitch describes, no special browser required.