R already has a general-purpose LLM evaluation framework in the tidyverse’s vitals. What it didn’t have was a security-specific one: labeled datasets of prompt-injection attempts, dangerous code, and credential leaks, plus harnesses that answer the questions security work actually asks.
Does my injection detector catch attacks without blocking legitimate queries?
Did tightening a guardrail reopen a hole it used to close? Which of two
configurations misses fewer threats? securebench evaluates any boolean
guardrail against labeled data and reports recall on attacks, precision on
benign traffic, and regressions case by case. as_vitals_scorer() turns any
guardrail into a vitals scorer, so the two packages work together rather than
competing.