measurement

Design signals that distinguish suspicious inputs, unauthorized proposals, blocked operations and confirmed exposure. Test response timing, evidence handling and monitoring health.

A guardrail that blocks everything scores perfectly on attacks. The scored answer measures both errors, tests with attacks the guardrail has not seen, and reports with intervals at the operating threshold.

Define current task authority and forbidden effects, test policy and agent behavior together, and report proposals, outcomes and useful completion with honest denominators and uncertainty.

Map effective authority, test component and handoff boundaries in authorized isolation, and report captured effects with coverage limits. Keep adaptive discovery separate from estimates of production risk.

Most broken defences were evaluated by their authors. The scored answer pins down the threat model, attacks the defence specifically, runs sanity checks, and reports robust accuracy with the conditions attached.

Define the success event and trial unit, reconcile unknown outcomes and report a rate whose uncertainty and retry claims match the actual assessment.

Calibrate a judge with meaningful reference labels, both error directions and correct audit weights. A high recall score alone does not explain its reported rate.

A falling aggregate can hide worse observed results in every input slice. Learn to inspect mixtures, missing outcomes, scorer changes and the release requirement.

Compare actual configurations with paired evidence, separate adaptive budgets and useful-task checks. A lower rate or significant test does not settle the release decision.