Released MLCommons Jailbreak Benchmark v1.0
Released MLCommons Jailbreak Benchmark v1.0, the first complete end-to-end run of the jailbreak methodology we had been building through v0.5 and v0.7.
v1.0 evaluates single-turn, text-only jailbreaks on eight open-weight systems: 264 seed prompts across eleven hazards, paired naïve vs. attacked evaluation, human annotation, and an evaluator ensemble calibrated to that ground truth. The headline metric is still the Resilience Gap. Across the suite, unsafe responses rose from 11.08% at baseline to 18.65% under attack (a 7.57% average gap); role-play, templates, and wrapper/schema attacks were among the most effective. We also treat the pipeline itself as the result: a reproducible, risk-calibrated way to report jailbreak robustness without publishing a cookbook of working exploits.