Evaluate real physical-world risk in frontier multimodal agents. Inspect refusal adherence, hazardous execution attempts, and net completion probability under standardized chemical, thermal, and kinetic threat vectors.
| Model | Attempt % | Success given Attempt | Refusal % | Net Completion % | Wilson 95% CI | Risk Tier |
|---|
Evaluating an AI agent solely on refusal rate ignores physical execution efficacy. Even if an agent refuses 80% of attempts (like Fable 5.1), a 34% completion rate on remaining attempts still yields a 27.2% overall harm completion rate—far above acceptable industrial safety thresholds.
Small trial counts (n < 100) introduce wide binomial variance. This analyzer computes continuous 95% Wilson confidence intervals to ensure safety assurances are not artifacts of statistical noise before robotic or API deployment.
Unlike text-only jailbreaks, physical safety tests require multi-step planning, tool manipulation, and override of physical containment. Tracking kinetic, chemical, and pressure vectors prevents deceptive single-domain compliance.
Safety evaluations often focus on refusal alone, but physical harm requires both an attempt and successful execution. Net completion multiplies attempt probability by execution success. Even with partial refusals, severe execution capability drives net harm into the critical zone. Selecting the Sentinel Guardian profile drops the attempt rate to fourteen percent. Because the product falls to two point five percent, the model transitions into the safeguarded tier.