TL;DR: Ask what property the claim covers and what evidence supports it. A finite evaluation of model behavior does not prove immunity across inputs, configurations and workloads. A narrower application property, such as rejecting an ungranted recipient on every send path, can have stronger mechanism evidence; verify its assumptions and the remaining consequences before approving a release.
How to approach it
Clarify the legitimate task, source authors, sensitive resources and actual audiences. Separate API message-role integrity, learned instruction priority and service authorization. Ask whether the vendor claims fewer observed failures, a particular operation prevented or universal model resistance. Those require different evidence and should not share one pass label.
A strong answer
A fictional mailbox assistant may summarize selected customer messages and prepare replies. A vendor reports zero observed injection successes and calls its model immune. The team's immediate question is whether an email author can expand what the assistant reads, change a reply's audience or distort information shown to the operator.
Observed model behavior
Versioned assessment
Cases, budget and grading
No observed violations
Within those conditions
Outcome, safe: Useful empirical evidence
Keep unknown inputs and workload changes outside the claim.
Scoped service permission
Current trusted grant
Actual resource and audience
All sending paths check
Verify deployment and bypasses
Outcome, safe: A specific enforced rule
State the assumptions; this does not prove draft accuracy.
Ask what was claimed
Zero observed model failures supports the tested result. It does not turn a finite input sample into universal immunity.
Instruction-hierarchy research investigates improved learned handling of conflicting instructions. It is evidence that model behavior can improve, not that API roles disappear or that all possible inputs are governed by a proved policy. Role metadata can remain intact while a model follows a lower-authority passage. Conversely, a bug that lets user content alter serialized API roles is a distinct implementation problem to fix.
Avoid claiming that finite studies prove every future defense can be defeated. The relevant review question is whether the evidence supports this product's claimed property. Defense-aware evaluation research motivates stronger assessment than a demonstration or a familiar public set, while retaining the tested scope and budget.
For the fictional assistant, inspect several independent claims:
| Claim | Evidence required |
|---|---|
| The model resisted the evaluated source material | Versioned cases, outcomes, budget and grading, including useful work |
| Only approved records can be read | Current grants from trusted inputs, complete mediation and negative-path tests |
| Replies reach only the approved audience | Checks on the actual recipient and content flow on every sending path |
| No remote image fetch occurs during rendering | The actual renderer and preview behavior, plus alternate paths within scope |
| Drafts remain accurate and appropriate | Task-specific quality and policy evaluation; resource checks alone do not establish this |
A domain allowlist enforces destination membership only under its implementation assumptions. A listed host may still contain an unauthorized audience. A human approval can help when it shows the actual immutable operation, but a standing grant may legitimately permit a bounded automatic reply. Neither a generic confirmation nor a reassuring model summary replaces the real permission check.
Read a zero result carefully. For independent comparable trials with one fixed failure probability, zero of 1,000 gives a one-sided 95% binomial upper confidence limit near 0.30%. A selected adversarial suite does not automatically satisfy that sampling model or represent future traffic. Keep the observed zero, its coverage and unknowns; do not turn it into proof of immunity or a daily incident forecast.
Choose a release scope supported by the evidence. An internal-draft pilot may remove a publishing capability while preserving useful review work, but private-data exposure and misleading drafts still need assessment. If a required grant check is missing, repair or remove that path. If remaining uncertainty concerns a supported content task, define the evidence, fallback and accountable decision instead of pretending a tool restriction solves language quality.
Test only owned or explicitly authorized systems with synthetic data, isolated accounts and controlled effects. Do not disable a production guard to demonstrate the claim's weakness.
What interviewers probe next
- "Would better models change the decision?" Better observed behavior can change the evidence, but new versions and routing require review; current resource and audience permissions still need enforcement.
- "What if the model cannot send anything?" Verify that restricted deployment, including previews and exports. Text disclosure or harmful advice can still matter without a sending tool.
- "What if a capability-based design is offered?" Examine the exact tracked property, trusted policy inputs and mediation coverage, then test supported tasks and paths outside the guarantee.
What if?
What if · interview follow-ups
Hold your answer as the constraints change
A fictional mailbox assistant is proposed for launch after a vendor reports zero observed injection successes. The team must assess read permissions, reply audiences and operator-visible drafts.
Start here
The vendor tested a thousand selected cases with no failures. Is a production failure probability below 0.3% established?
What population and independence assumptions support the calculation?
Not from the count alone. The simple binomial confidence limit requires independent comparable trials with a fixed probability, and applying it to production requires a representative sampling argument. Record the clean selected-set result without inventing a population guarantee.
Carry it forwardA test count does not establish its sampling model.
Weak answerUsing a confidence formula to certify unrelated production traffic.
Constraint changes
An email author changes the proposed recipient, but the service rejects it. Did the entire system fail?
Which properties held and which may have failed?
The model may have followed an unauthorized source instruction, while the recipient check held for the tested send. Inspect whether sensitive material reached another audience through drafts or alternate paths. Report each observation and its scope instead of one undifferentiated success or failure label.
Carry it forwardComponent outcomes need separate evidence.
Weak answerErasing a model failure or a working service check.
Constraint changes
The team proposes a draft-only pilot. Is that automatically acceptable?
What can the operator still read or rely on?
No. Verify the absent publishing paths and current read grants, then assess sensitive text exposure, draft quality and the actual review workload. A narrower scope can reduce risk and preserve value, but the remaining consequences still need an explicit decision.
Carry it forwardReduced capability changes the review; it does not finish it.
Weak answerEquating no sending tool with harmless output.
Now defend it
What would your recommendation say?
Name the supported task and model configuration, the precise service properties and their assumptions, the observed and unresolved outcomes, and valid-work results. Recommend a bounded scope with owners, fallback behavior and conditions that reopen the review; do not repeat an unsupported immunity claim.
How it's scored
This exercise rewards a precise assurance claim and a release decision traceable to evidence.
Weak: Accepts a clean demonstration as immunity, or argues that no scoped security property can ever be established.
Hire: Separates model behavior from application permissions and asks for versioned outcomes, coverage and legitimate-work evidence.
Strong Hire: Examines the assumptions behind enforced properties, handles statistical limits correctly and proposes a useful bounded release with clear reopening conditions.
Common mistakes
- Treating API-role metadata and learned instruction-following behavior as the same mechanism.
- Calling every detector miss an actual harmful outcome.
- Making an allowed hostname or a confirmation click stand in for audience authorization.
- Assuming an internal or text-only feature has an automatically low impact ceiling.