Scaffold Effect Simulator

Build Your Harness

Every knob below changes both the expected token bill and the chance the task gets solved. Adjust and the simulation reruns instantly.

Base solve probability / attempt30%

Max retries3

Context strategy

Tool calls / attempt4

Verification passes1

Tokens / attempt

Tokens / tool call

Tokens / verification

Live Simulation — 1,000 Monte Carlo Trials

Each trial runs the agent loop with your settings. Verification catches bad solutions (base 30% detection, each pass adds detection but costs tokens); caught failures retry, uncaught ones ship wrong answers.

—Solve rate

—Median tokens / trial

—Tokens / solved task

Distribution of tokens burned across 1,000 trials.

The Agent Loop

Node size pulses in proportion to where your tokens actually go under the current settings.

Compare Harnesses

Pin your current config on the Pareto chart. Up and to the left is better: higher solve rate, fewer median tokens. The efficient frontier is drawn in amber.

Cheap & Greedy (30% solve, 1 retry, fresh context, 2 tools, 0 verification): dirt-cheap per trial, but with no verification it ships whatever the model produces — solve rate stays pinned near base ability.

Verify Everything (30% solve, 6 retries, summarized context, 6 tools, 3 verification passes): burns several times the tokens, but catching bad solutions converts retries into real wins — solve rate can more than double.

Check Your Intuition

Five quick questions on retry math, verification tradeoffs, and expected cost.

Why this matters

Two teams can deploy the same model and see wildly different unit economics. The scaffold — how many times you retry, what context you carry, whether you verify before shipping — is a design surface as important as model choice. This simulator is a toy, but the shape of the tradeoff is real: verification and retries buy solve rate with tokens, and context strategy decides how steep that bill grows.