TL;DR: Draw the flow: customer, chat service, context assembler, model, knowledge base, order tool, renderer. The top risks are order data for the wrong customer (authorization must use the session's customer ID in code, never an ID the model supplies), injected instructions from the customer or from knowledge-base content steering the bot, unsafe rendering of its replies, and cost abuse. Prompt instructions and classifiers lower rates; the order tool's ownership check is what holds the boundary.
How to approach it
Ask three questions first: are customers signed in when they chat, who can edit the knowledge base, and can the bot take actions beyond lookup (refunds, address changes)? The answers change the threat model more than anything about the model itself. Then sketch the data flow and say you will start with the order tool, because it is where the bot touches data that belongs to one customer.
A strong answer
Customer
Signed in
Chat service
Holds the session
Model
Order tool
Checks ownership
Orders DB
Also in the picture
Knowledge base
- Staff-edited
The model asks, the tool decides
The model may request order 4412 because a customer asked or because injected text told it to. The tool ignores who asked and checks the order against the session's customer ID.
Assets and adversaries. Assets: customers' order and personal data, the company's brand (what the bot says), and model spend. Adversaries: a signed-in customer trying to see other people's orders or extract policy exceptions; anyone who can influence knowledge-base content; and anyone who can script the chat endpoint.
Threats, by boundary.
| Boundary | Threat | Control that holds | Control that helps |
|---|---|---|---|
| Model to order tool | Bot fetches another customer's order (asked directly or via injection) | Tool takes the customer ID from the authenticated session, not from model arguments; checks ownership on every call | Prompt instructions to stay on topic |
| Customer to model | Direct injection to extract the system prompt or make policy promises ("approve my refund") | Nothing secret in the prompt; actions like refunds go through policy code, not the model's say-so | Input classifiers, refusal training |
| Knowledge base to model | A poisoned or careless article steers answers | Edit permissions and review on the knowledge base; content provenance | Injection detection on retrieved text |
| Model to renderer | Reply contains markup or a link that leaks data | Render as text or sanitised Markdown; no external images | Output filtering |
| Customer to service | Scripted abuse runs up model cost | Per-user rate and token limits | Bot detection |
| Logs | Transcripts hold personal data with broad access | Redaction, retention limits, restricted access |
What would change the answer. If the bot can issue refunds, the refund tool needs its own limits and a policy check in code, and large refunds need human approval shown from the call's parameters. If anonymous visitors can chat, the order tool must require verification before any lookup.
What interviewers probe next
- "Customers ask for orders by number. Can't the model pass the number through?" It can pass the number; it must not be trusted to pick the customer. The tool resolves the order and checks it belongs to the session's customer.
- "How would you test this before launch?" Automated cases that request other customers' orders through direct and injected prompts and expect denial from the tool, plus a red team pass on refunds and policy promises.
- "Legal says the bot must never promise refunds. How?" Any commitment comes from policy code; the model's text is labelled informational, and an output check catches promise language as a backstop.
What if?
What if · interview follow-ups
Hold your answer as the constraints change
A bank launches a chatbot that answers account questions and can look up recent transactions for the logged-in customer.
Start here
What does the flow look like?
What feeds the model's context?
Customer, chat service, context assembler, model, knowledge base, transaction tool and renderer. Each arrow is a place to ask what crosses and who controls it.
Carry it forwardMap the flow including context assembly.
Weak answerModelling only customer and model.
Constraint changes
What is the top risk?
Whose ID does the tool use?
Transaction data for the wrong customer. Authorisation must use the session's customer ID in code, never an ID the model supplies, because the model can be persuaded to ask for anything.
Carry it forwardTools authorise with the session identity.
Weak answerPassing a model-chosen customer ID.
Constraint changes
Where can injected instructions come from?
Who writes transaction descriptions?
The customer directly, and indirectly through knowledge base articles or transaction descriptions written by merchants or other parties. Treat all of it as untrusted.
Carry it forwardIndirect injection through data fields.
Weak answerAssuming only the customer can inject.
Constraint changes
What about the output?
What could a rendered link do?
Render as safe text without remote images or arbitrary links, so injected content cannot exfiltrate data or phish the customer. Avoid giving advice the bank cannot stand behind.
Carry it forwardSafe rendering stops exfiltration.
Weak answerRendering model Markdown with images.
Constraint changes
What else belongs in the model?
What would an incident investigation need?
Logging with privacy controls, abuse and cost limits, human handoff for complex requests, regulatory disclosure that customers are talking to AI, and evaluations for leakage and injection measured as rates.
Carry it forwardLogging, limits, handoff, disclosure and evaluation.
Weak answerLaunching without regression tests.
Now defend it
What would the threat model conclude?
Authorise the transaction tool with the session's customer ID in code; treat customer text, knowledge articles and transaction descriptions as untrusted; render output safely without remote images; keep the model tool-limited and read-only; log with privacy controls; limit cost; provide human handoff and AI disclosure; and test leakage and injection as rates.
How it's scored
Weak: Focuses on jailbreaks and content filters; lets the model pass the customer identifier to the order tool.
Hire: Puts ownership checks in the tool using the session identity, covers injection from both the customer and the knowledge base, and limits cost.
Strong Hire: Separates holding controls from helping ones, covers rendering and logs, and shows how the model changes when refunds or anonymous chat are added.
Common mistakes
- Trusting a customer ID that came from the model's tool arguments.
- Treating knowledge-base content as trusted because staff wrote it.
- Putting discount codes or internal policies in the system prompt.
- Ignoring transcript storage as a data store with its own risks.