Why routing beats lock-in
Single-vendor procurement freezes yesterday's price/quality curve. A router re-scores every request, so when a new model drops 10x cheaper next quarter, you capture the gain without a migration project. Model choice becomes a runtime decision, not a contract decision.
The scoring formula
Each model gets a weighted score:
S = w_c·cost + w_l·speed + w_k·control + w_h·hosting + taskFit
Weights come from your sliders (normalized to sum 1). Task fit is a per-task bonus: a small self-hosted model may win PII work even if a frontier model is "smarter."
ROI is task-specific
Frontier models can be 30–100x the cost per token of small open models. If a task's accuracy plateaus at 8B parameters, the extra spend is pure waste. Routers measure quality-per-dollar per task, and route the boring 80% of traffic to cheap models.
What enterprises score
Cost ($/1M tokens), latency (time-to-first-token & tokens/sec), controllability (fine-tuning, system-prompt adherence, structured output), and hosting (API-only vs. VPC vs. on-prem weights). Regulated workloads often hard-require self-hosting — that's why the PII toggle reshuffles the leaderboard.
How a router works in practice
A thin gateway sits in front of all model APIs. Each request carries a task tag; the router looks up per-task evals (accuracy, refusal rate, latency p95, $/req) and applies your weight policy. Fallback chains handle outages: if Frontier-XL times out, the request re-routes to Mid-Pro automatically — vendor incidents stop being your incidents.
Procurement shift
Old world: 3-year enterprise license, one vendor, quarterly business reviews. New world: models are interchangeable line items re-bid continuously by software. Negotiating leverage comes from the credible ability to shift traffic in minutes. Teams that publish internal per-task leaderboards routinely cut inference spend 40–70% with no quality loss.