Released AILuminate Jailbreak v0.5: measuring the Resilience Gap
Safety scores are not the same as security. Released AILuminate Jailbreak v0.5 from MLCommons, the draft benchmark that asks a harder question: what happens when someone tries to bypass the guardrails?
We start from a naïve safety baseline (AILuminate v1.0 for text, MSTS for vision-language), then apply known jailbreak families and measure the Resilience Gap, the drop in non-violating rate under attack, with the same evaluator and five-tier grades. Across 39 text-to-text systems, none matched their safety grade once jailbroken; 35 dropped a grade, with an average safety-score reduction of about 20 points. The benchmark is a measuring instrument inside an improvement loop, not a trophy.