Input tokens
Everything the model reads per API call: system prompt, tool definitions, full conversation history, tool results. In agents this is re-sent on every call, so it compounds turn after turn.
The four metrics that matter
Output tokens
Everything the model writes: reasoning, tool-call arguments, final answers. Priced ~5× higher than input, but usually far smaller in volume for agents.
Tool calls
Each tool invocation = an extra model round-trip. More tools per turn means more calls, and every call re-reads the whole context. Tool results also get appended to history — growing all future inputs.
Latency
Wall-clock time per turn ≈ (API round-trips × overhead) + (output tokens ÷ generation speed) + tool execution time. Output tokens are the slow part — models generate tens of tokens per second but read thousands.
The core insight: a chat app’s context grows linearly, but an agent’s billed input grows roughly quadratically with conversation length — because each of the N calls re-sends a history that is itself growing. Observability dashboards exist to catch this before your invoice does.
Live session simulator
Each pair of 3D bars is one turn: cobalt = input tokens, tangerine = output tokens. Drag to orbit. Move the sliders and watch how fast input dwarfs output.
Drag with mouse or finger to rotate · bars re-grow when you change the workload
—Total input tokens
—Total output tokens
—API calls / tool calls
—Est. session latency
—Session cost (Claude Sonnet pricing)
Presets
The math, worked by hand
How the simulator counts tokens
Fixed assumptions: user message = 150 tok, each tool call the model writes = 120 tok (output), each tool result returned = 700 tok (input on the next call).
- History before turn i = (i−1) × (user 150 + output/turn + tools×(120+700)).
- Turn i, call 1 input = system prompt + history + new user message.
- Each tool round adds another API call whose input = call-1 input plus the tool calls and results so far this turn. A turn with T tools =
T+1API calls. - Turn output = T×120 (tool-call args) + final answer tokens.
Pricing (Claude Sonnet class)
$3.00 / 1M input · $15.00 / 1M output · cached input reads $0.30 / 1M. Cost = in×3/10⁶ + out×15/10⁶.
Latency model
≈ 0.8 s fixed overhead per API call + output tokens ÷ 60 tok/s generation + 0.5 s per tool execution.
Example: 8 turns, 2 tools/turn, 6k system prompt, 400-tok answers
| Turn | Input tok | Output tok | Cost |
|---|
Turn 1 reads ~— input tokens; turn 8 reads ~— — same question size, ~—× the input bill. That growth curve is what session-level token observability is for.
What to alert on
Input/output ratio drift
Healthy agents often run 10–50× more input than output. A sudden jump means runaway context — a loop appending junk, or tool results that are too fat.
Tool calls per session
Spikes signal retry loops or a confused agent hammering the same tool. Set a hard budget (e.g., 25 calls/session) and alert at 80%.
Cost per completed task
Tokens are the input; dollars-per-outcome is the KPI. A $0.09 session that solves the ticket beats a $0.02 one that doesn’t.
Cache hit rate
With prompt caching, stable prefixes (system prompt, tool defs, old history) can be re-read at ~10% of the price. Flip the caching toggle above — same tokens, a fraction of the cost.