Multi-Tier Model & Prefix Cache Router

Benchmark inference cascades and prompt caching economics across flagship reasoning (Astra), balanced execution (Sol), and high-throughput routing (Luna).

Monthly Cost
$1,142
-82.4% vs all-Astra ($6,480)
Effective P50 Latency
240 ms
Astra alone: 1,450 ms (3.8x faster)
Cache Cost Reduction
$890
Prompt prefix caching benefit
Quality Index
96.8 / 100
Weighted output correctness

Cascade Traffic Allocation & Pricing

OpenAI GPT-6 Universe Specifications
GPT-6 Luna 55%
Routing, classification, light parsing
$0.15 / $0.60 per 1M (Cached $0.038)
$98 / mo
GPT-6 Sol 35%
General agent logic, code & extraction
$0.80 / $3.20 per 1M (Cached $0.20)
$396 / mo
GPT-6 Astra 10%
Deep reasoning, edge cases, validation
$5.00 / $15.00 per 1M (Cached $1.25)
$648 / mo
Simulated Stream Inspection (Latest 5 Inbound Requests) Simulated stream active.
Req ID Task Intent Prefix Cache First Pass Escalation Path Latency Cost

Production TypeScript Router (Drop-In SDK Cascade)

Architectural Advantage

The GPT-6 family introduces a tri-tier operational matrix: GPT-6 Astra provides frontier reasoning, while GPT-6 Sol and GPT-6 Luna bring that foundational architecture into fast, affordable production tiers.

By coupling fast first-pass validation with shared prompt prefix caching, teams avoid routing 100% of volume through flagship models while preserving near-identical accuracy.

Optimization FAQ

How does Prompt Prefix Caching work in GPT-6?

Prefix caching reuses precomputed attention states for static parts of the prompt (system guidelines, schemas, few-shot examples, large codebases). GPT-6 yields up to 75-80% discount on cached input tokens with near-zero time-to-first-token (TTFT).

What criteria triggers escalation from Luna to Sol or Astra?

Production routers use three signals: token confidence scores, structured JSON validation errors, and regex assertion tests. If Luna produces an invalid schema or indicates low confidence, the request seamlessly escalates to Sol or Astra with zero customer impact.

Can this configuration be exported to LangChain or LiteLLM?

Yes. The downloadable configuration file includes exact fallback rules, model IDs, token limits, and cache keys ready for LiteLLM router proxy, custom edge workers, or Python SDK pipelines.

Trace assigned routing shares and token-cost assumptions

Read the explanation

This saved routing calculator treats Luna and Sol pass sliders as unconditional request shares, not conditional success probabilities. Its balanced preset sets fifty-five percent and thirty-five percent, leaving ten percent for Astra. Bars compare fifty-five and thirty-five assigned percent at nine pixels per percentage point. The exported routing shares sum to one under this source rule; the tool does not call any model or measure actual pass rates. Cached prompt tokens are prompt length times cache hit fraction; fresh input uses the complementary fraction. With an illustrative one thousand prompt tokens and the balanced assigned sixty-eight percent cache fraction, six hundred eighty are billed in the cached-input branch and three hundred twenty in the fresh branch. Bars show assigned token counts at zero point seven pixels per token. Source price coefficients are fixed assumptions; this does not establish current provider prices or actual cache reuse. The source caps the combined Luna plus Sol share at ninety-five percent by reducing Sol. An illustrative eighty percent Luna with forty percent requested Sol therefore becomes fifteen percent Sol and five percent Astra. Bars compare the requested forty with the assigned fifteen at twelve pixels per percentage point. Its latency label is an arithmetic weighted average of authored tier latency numbers; this is not a statistically derived median or benchmark. Preserve the clamp and modeling limits.

Enjoy this tool? Build your own with Super