How does this benchmark calculate efficiency multipliers and preference splits between terminal agents and inline assistants?
This interactive workbench models workflow tradeoffs between autonomous command-line interface (CLI) agents and single-file inline code assistants using synthetic heuristics. Rather than reporting live benchmark executions or empirical developer survey outputs, the client-side JavaScript applies algebraic formulas to codebase scale, test run frequency, and guardrail settings to project an efficiency multiplier between 1.8x and 3.6x alongside an estimated preference split.
All displayed efficiency gains, developer preference percentages, radar charts, and execution logs are pre-scripted UI simulations driven by simplified client-side formulas. They do not represent physical hardware execution traces, real repository analyses, or validated academic research.
Select 'Git 3-Way Merge Conflict Resolution' in the Target Task Pipeline dropdown. The simulation automatically triggers after previous runs finish, updating the simulated terminal panes to step through git conflict triage commands, while recalculating telemetry based on the active codebase scale and guardrails.
The tool computes the simulated efficiency multiplier using the linear formula: mult = 2.4 + (scale / 250) * 0.4 + (freq * 0.1), clamped between 1.8 and 3.6. Guardrail selections add fixed offsets (+0.1 for standard confirmation, +0.3 for permissive execution).
The developer preference distribution uses an analogous heuristic: pAgent = 68 + (freq * 2) + (scale > 100 ? 5 : 0), with a +2 adjustment for standard confirmation, bounded within 60% to 88%. These parameters illustrate design assumptions rather than real-world sampling.
The side-by-side terminal views render predefined sequence arrays on an interval timer. Because state tracking marks isRunning as true during active intervals, subsequent user adjustments queue visual telemetry updates while running traces resolve their scheduled timeouts.
The source initializes repository size eighty five, testing frequency four and standard guardrails. Its telemetry computes two point four plus size over two hundred fifty times point four, plus frequency times point one and a point one guard adjustment. Three point zero three six rounds to three point zero. Bars compare that calculated value with the initial state literal two point eight, which initialization overwrites. These are assumed multipliers, not measured benchmarks. Default preference is sixty eight plus two times the frequency four plus two for standard guardrails, yielding seventy eight percent. The complementary value is twenty two. Above one hundred thousand lines the formula adds five points, and the result is clamped between sixty and eighty eight. Bars use five pixels per percentage point. These percentages are generated by the interface, not survey evidence about users or actual model choice. The token reduction label is round of thirty plus repository size over five hundred times twenty. At eighty five this becomes thirty three percent. Changing token budget alone leaves this label unchanged, because tokens are stored but not used by that formula. The animated terminals replay authored steps; they do not execute the shown tests. This source therefore supports a parameterized teaching demo, not claims of validated agent superiority.