TL;DR: Detect through product-specific signals, then establish which reads, releases or actions actually occurred. Link identity, context provenance, authorization and effect records without confusing correlation with cause. Contain credible ongoing harm, scope related events and report uncertainty while testing the explanation and repair.
How to approach it
Start with the protected asset and the behavior that violates its requirement. Ask who can supply input, which services the assistant can use and who can read its outputs. Separate the alert, the confirmed event and the causal hypothesis; those stages support different decisions.
A strong answer
Use an original fictional research assistant that reads external pages and publishes reports into project spaces. The tester controls one approved page in an isolated environment. The requirement is that restricted research cannot be published to a space whose readers lack access.
Design signals around the product. Collect publication events, changes in readers, authorization denials, unusual destinations, source-version relationships and user reports. Model or guardrail alerts can add context. A token spike can indicate abuse, a loop or legitimate work; it is not an injection diagnosis.
A planted marker reaching a prohibited receiver can support a disclosure finding when the marker’s provenance and visibility are controlled. A marker already supplied by the user, or found only in an internal log, does not establish that disclosure. Canaries complement ordinary service evidence; they are not a prerequisite for detecting every leak.
Signal
Report, anomaly or control event
Event evidence
Identify actual reads and effects
Competing explanations
Check identity, context and state
Response decision
Contain, scope and test repair
Verify the event
An unusual destination is a lead. An authoritative delivery record and recipient evidence establish a narrower, stronger fact.
Reconstruct a supported event chain. Capture the request, authenticated user and service identity, destination, relevant source versions and operation ID. Distinguish the model’s proposal from gateway admission, downstream commit and the readers of any stored report. Check alternate publication paths and jobs that continued after the foreground request ended.
The context record helps establish what information was available. It does not expose the model’s internal reasoning or prove which input caused an action. Preserve the provider identifiers actually available rather than pretending a mutable model alias identifies immutable weights.
Investigate competing explanations. The external page may have influenced publication, but an authorized user request, stale project selection, overbroad service credential or ordinary application bug can produce similar symptoms. Inspect the failed enforcement boundary as well as the content. Direct user misuse can also be a security incident; it does not become harmless merely because the request was explicit.
Use controlled, authorized comparisons with synthetic data and receivers. Removing a page may also remove facts or change the workflow, so record the intervention. A verified historical disclosure remains actionable if the current configuration cannot reproduce it. Do not generate another real disclosure just to strengthen the report.
Contain and scope in parallel. Disable the affected publication path or enforce a narrower audience. Include background jobs and retained drafts. Preserve relevant evidence through the approved incident process without delaying urgent containment for a perfect snapshot.
Search by source version, action implementation, policy revision and affected destinations to find related cases. These matches identify potentially affected activity. Confirm which data reached which readers and track missing receipt coverage separately. Involve the designated privacy and communications owners with the facts and uncertainties.
Write a useful triage record. For this fictional case, classify four statements before choosing the next action:
| Statement | Classification |
|---|---|
| The project service records a restricted report accessible to an unauthorized collaborator | Confirmed boundary failure |
| The external page caused the publication | Hypothesis requiring further evidence |
| Twelve sessions assembled the same page version | Related exposure set, not twelve confirmed disclosures |
| No delivery receipt exists for a second publication job | Unresolved effect |
The next action is to contain publication, preserve the records and reconcile the second job. A root-cause meeting is not a prerequisite to those steps. The broader preparation and recovery framework is covered in AI incident response.
What interviewers probe next
- “Full prompt logging is prohibited here. Can you investigate?” Use approved metadata, source versions, service records and permitted content references; describe what cannot be reconstructed.
- “The model says the page instructed it.” Treat that explanation as a claim, then test it against the event chain and alternatives.
- “No alerts fired.” Review detection coverage and collector health; silence does not establish that no event occurred.
What if?
What if · interview follow-ups
Hold your answer as the constraints change
In an original fictional exercise, a research assistant appears to have published a restricted report to a shared project space after reading an external page.
Start here
What if the page appears immediately before the publish proposal?
Timing supplies a lead.
Preserve the source version and assembled context, but do not call that sequence proof of injection. Confirm the publication and actual readers, inspect the authorization decision, and compare alternative explanations such as an explicit user request, stale destination state or an application identity bug.
Carry it forwardExposure and timing are not a complete causal argument.
Weak answerTreating the last source in the prompt as the proven cause.
Constraint changes
What if only the project-space audit log remains?
Use the evidence that survives.
That log may still establish publication, authorizing identity, time and readership. It may support containment and part of the affected scope even if the model context is unavailable. Record the missing input evidence, seek permitted related records and keep the mechanism provisional.
Carry it forwardPartial evidence can support useful, limited conclusions.
Weak answerCalling the whole incident unknowable without full prompts.
Constraint changes
What if switching to drafts leaves those drafts visible to collaborators?
Check every materialized output.
Then draft-only mode does not remove the relevant disclosure path. Restrict publication or draft readership at the service that enforces it, and review retained drafts already accessible. Do not rely on a final send action being disabled when an earlier store is itself a release.
Carry it forwardContain the reader boundary, including stored artifacts.
Weak answerAssuming an unsent artifact has no audience.
Now defend it
What would you put in the first incident update?
State the verified event and recipients, current containment, potentially affected scope, unresolved effects and hypotheses, and the next check with an owner. Keep confidential evidence behind restricted references. Do not delay the response until a complete causal narrative is available.
How it's scored
Weak: Calls every anomaly injection or infers cause from the most recent document.
Hire: Connects identities, context sources, authorization and actual effects; contains and scopes with evidence limits stated.
Strong Hire: Handles missing logs and pending jobs, tests alternative causes safely, distinguishes potentially exposed from confirmed recipients, and defines a usable response update.
Common mistakes
- Waiting for repeatability before acting on a confirmed disclosure.
- Treating draft-only mode as safe without checking draft readers.
- Counting every source match as an affected customer.
- Recording a missing effect receipt as a successful denial.