Drag to rotate · Blue cubes are tokens in, gold coins are revenue out — the gauge is your gross margin
—
AI cost / user / mo
—
Gross profit / user
—
Gross margin on AI spend
The formula behind the machine
cost_per_user = actions × tokens_per_action
× (cost_per_1M / 1,000,000)
Example (defaults above):
300 actions × 8,000 tok × $6/1M
= 2.4M tokens × $0.000006
= $14.40 per user per month
margin = ($20 − $14.40) / $20 = 28%
28% gross margin is dangerously thin for software — investors expect 70–80%+. That's the whole reason "token ROI" dashboards are suddenly in demand: many AI features quietly run at software prices with utility-company margins.
Five levers when margin is thin
- Model routing: send simple requests to models 10–20× cheaper; reserve frontier models for hard tasks. Often the single biggest win.
- Prompt caching: reusing long system prompts can cut input token cost dramatically on repeat calls.
- Output discipline: shorter, structured outputs. Output tokens usually cost several times more than input tokens.
- Usage-based pricing: align price with cost — heavy users pay more, so no user is margin-negative.
- Batching & async: batch APIs are often ~50% cheaper when the user doesn't need an instant answer.
What to actually instrument
- Cost per feature, not per app: tag every API call with the feature name. One feature is usually 80% of spend.
- Cost per retained user: tokens spent on users who churn next month are pure loss — join billing data with retention cohorts.
- Margin percentile, not average: your p95 user might cost 30× your median user. Averages hide the users bleeding you.
- Revenue attribution: A/B the AI feature off for a small cohort. If conversion and retention don't move, the honest ROI of those tokens is zero — no dashboard will tell you that; only the experiment will.