Agent Observability: Tokens, Tools & Cost

The four metrics that matter

Input tokens

Everything the model reads per API call: system prompt, tool definitions, full conversation history, tool results. In agents this is re-sent on every call, so it compounds turn after turn.

Output tokens

Everything the model writes: reasoning, tool-call arguments, final answers. Priced ~5× higher than input, but usually far smaller in volume for agents.

Tool calls

Each tool invocation = an extra model round-trip. More tools per turn means more calls, and every call re-reads the whole context. Tool results also get appended to history — growing all future inputs.

Latency

Wall-clock time per turn ≈ (API round-trips × overhead) + (output tokens ÷ generation speed) + tool execution time. Output tokens are the slow part — models generate tens of tokens per second but read thousands.

The core insight: a chat app’s context grows linearly, but an agent’s billed input grows roughly quadratically with conversation length — because each of the N calls re-sends a history that is itself growing. Observability dashboards exist to catch this before your invoice does.

Live session simulator

Each pair of 3D bars is one turn: cobalt = input tokens, tangerine = output tokens. Drag to orbit. Move the sliders and watch how fast input dwarfs output.

Drag with mouse or finger to rotate · bars re-grow when you change the workload

—Total input tokens

—Total output tokens

—API calls / tool calls

—Est. session latency

—Session cost (Claude Sonnet pricing)

Turns in session 8

Tool calls per turn 2

System prompt + tool defs 6,000 tok

Assistant output per turn 400 tok

Prompt caching on (cached reads at 10% price)

Presets

The math, worked by hand

How the simulator counts tokens

Fixed assumptions: user message = 150 tok, each tool call the model writes = 120 tok (output), each tool result returned = 700 tok (input on the next call).

History before turn i = (i−1) × (user 150 + output/turn + tools×(120+700)).
Turn i, call 1 input = system prompt + history + new user message.
Each tool round adds another API call whose input = call-1 input plus the tool calls and results so far this turn. A turn with T tools = T+1 API calls.
Turn output = T×120 (tool-call args) + final answer tokens.

Pricing (Claude Sonnet class)

$3.00 / 1M input · $15.00 / 1M output · cached input reads $0.30 / 1M. Cost = in×3/10⁶ + out×15/10⁶.

Latency model

≈ 0.8 s fixed overhead per API call + output tokens ÷ 60 tok/s generation + 0.5 s per tool execution.

Example: 8 turns, 2 tools/turn, 6k system prompt, 400-tok answers

Turn	Input tok	Output tok	Cost

Turn 1 reads ~— input tokens; turn 8 reads ~— — same question size, ~—× the input bill. That growth curve is what session-level token observability is for.

What to alert on

Input/output ratio drift

Healthy agents often run 10–50× more input than output. A sudden jump means runaway context — a loop appending junk, or tool results that are too fat.

Tool calls per session

Spikes signal retry loops or a confused agent hammering the same tool. Set a hard budget (e.g., 25 calls/session) and alert at 80%.

Cost per completed task

Tokens are the input; dollars-per-outcome is the KPI. A $0.09 session that solves the ticket beats a $0.02 one that doesn’t.

Cache hit rate

With prompt caching, stable prefixes (system prompt, tool defs, old history) can be re-read at ~10% of the price. Flip the caching toggle above — same tokens, a fraction of the cost.

Repeated-context token arithmetic and cache assumptions

Your browser does not support video.

Read the explanation

The source token model assumes eight turns, two tools per turn, six thousand initial context tokens and four hundred final output tokens per turn. Each user message adds one hundred fifty, each tool call one hundred twenty and each tool result seven hundred. History grows by two thousand one hundred ninety per turn. Three calls reread context within each turn, so first-turn input is twenty thousand nine hundred ten and eighth-turn input sixty six thousand nine hundred. Bars span sixty two point seven three and two hundred point seven at point zero zero three pixels per token. Summed input is three hundred fifty one thousand two hundred forty, output five thousand one hundred twenty, with twenty four modeled calls. No actual API calls occur. The source assigns three dollars per million fresh input tokens, fifteen per million output and thirty cents per million cached input. Those are fixture constants, not verified current provider prices. Its cache heuristic marks three hundred thirty four thousand five hundred twenty input tokens cacheable and sixteen thousand seven hundred twenty fresh. Total cost is one point one three zero five two without caching and point two two seven three one six with caching. Bars span about two hundred three point four nine and forty point nine two at one hundred eighty pixels per dollar. The cached-input price is ninety percent lower, but total modeled cost falls about seventy nine point nine percent because output and fresh input still cost money. Token counts themselves do not change. The light preset uses four turns, zero tools, two thousand context and three hundred output per turn. Inputs total eleven thousand three hundred and outputs twelve hundred, represented by two hundred three point four and twenty one point six pixels at point zero one eight per token. The assumed latency is four calls times point eight seconds overhead plus twelve hundred output tokens divided by sixty tokens per second, giving twenty three point two seconds. It is an estimate, not measured throughput. The source installs arithmetic controls and creates the worked table before constructing Three. With missing Three, clicking a preset still updates captions before its bar rebuild fails on an undefined object. This distinction preserves real local arithmetic behavior without claiming a working 3D chart, actual billing, exhaustive tool checks or public deployment.