AI Agent Database Task Eval Matrix

Benchmark Suite

Agent Models Under Test

Claude Code 3.7 Codex CLI Open Code Agent

Overall Accuracy Rate 83.3% 5 of 6 target SQL assertions passed

RLS Security Pass Rate 66.7% 1 vulnerability detected (Infinite Loop)

Average Execution Latency 1420ms Per agent trace synthesis

Top Scoring Agent Claude Code (3.7) 100% precision on RLS & DDL

📊 Execution Log Matrix (6 Tasks) Showing all agents

Agent	Task Name	Category	Status	Latency

🔍 SQL / Security Diff Inspector eval-rls-01

Target Benchmark Tenant-Isolated RLS Policy

Agent Evaluated Claude Code (3.7 Sonnet)

Error Category None (Exact Match)

Generated SQL Output vs Benchmark Assertion

Security & Optimization Analysis

Policy correctly extracts org_id from JWT context and casts to UUID without subquery overhead or recursive loop risks.

Model Benchmark Comparison

RUNNABLE EVALUATION PROOF

Overall Accuracy Rate: 83.3% | RLS Security Pass Rate: 66.7% | Average Latency: 1420ms | Top Agent: Claude Code (3.7 Sonnet) | Total Executed: 6