Choose a measured operating point for the exchange.
Parse labeled probe and classifier traces, quantify selective escalation, and defend a two-stage monitoring policy with observed recall, precision, and compute.
Paper reference, not reproductionThe source reports about 5.5% escalation, a 0.55/0.45 blend, and roughly 3.5% relative compute in one deployment. This workbench evaluates your data and does not treat those values as universal targets.
Each point recomputes the cascade on the parsed labels. The vertical line is the active route threshold.
RecallPrecisionEscalation
No evaluated policy yet.
Observed policy result
Recall—TP / positives
Precision—TP / flags
Escalation—routed / rows
Relative compute—vs classify every row
False negatives—positive rows missed
Routing ruleProbe score must meet the route threshold before classifier cost is spent.
Decision ruleRouted rows blend both signals; unrouted rows retain the probe score.
Constraint selector
Find the highest route threshold that satisfies both measured constraints.
Evaluate labeled traces before selecting a policy.
The aggregate only earns trust when every row is inspectable.
Review which exchanges paid for the second stage, how signals blended, and where labels disagree with the final decision.
ID
Context
Probe
Classifier
Label
Routed
Final score
Decision
Outcome
No evaluated rows.
Read the policy without turning evidence into a guarantee.
The workbench computes validation-set behavior. Deployment drift, adaptive attacks, and unlabeled edge cases remain outside this result.
More rows purchase the second-stage classifier. That can recover positives whose probe scores sit below a stricter threshold, but it increases compute and can still introduce false positives depending on the blend.
The routing threshold decides when to buy more evidence. The decision threshold determines whether the available evidence is sufficient to flag. Conflating them hides an important operational tradeoff.
It lets scores reflect the relationship between request and unfolding response rather than treating isolated input or output fragments as independent evidence.
Preserve the measured policy.
The export contains settings, assumptions, aggregate metrics, the threshold sweep, and row-level decisions.
Measured policy
Threshold 0.57 meets both constraints.
Recall83.3%
Precision100.0%
Escalation66.7%
Relative compute68.4%
Observed consequence: two additional rows reached the classifier and one labeled positive was recovered while escalation stayed below 75%.