Skip to content
AI Security

Interview question · Adversarial ML & Model Privacy

How do training data extraction and memorisation work in LLMs, and how do you reduce them?

Language models are trained to predict text, and sometimes the best prediction is the exact text they saw. Rare, repeated and sensitive strings are the risk.

mediumEssentialNewPreparation tags: Google DeepMind, OpenAI, AnthropicUpdated Nov 2026

Language models are trained to predict text, and sometimes the best prediction is the exact text they saw. Rare, repeated and sensitive strings are the risk.

Make your first attempt

What would you say before reading the answer?

  1. 01 State the assumptions your answer depends on.
  2. 02 Explain the mechanism and choose a control.
  3. 03 Name a condition that would change your decision.

The worked answer includes an approach, scoring criteria and follow-up questions. Compare the reasoning with your own.

signing in unlocks more free answers per topic

What if · the follow-ups in this answer

The answer works through 6 follow-ups like an interviewer would

A company fine-tunes a language model on support transcripts that contain customer names, addresses and some account numbers, and plans to offer it to all customers.

  1. Why might the model reveal transcript details?
  2. What if the model is a production chat model, not a small research one?
  3. What should be done before fine-tuning?
  4. What controls help after training?
  5. What if one customer's account number is found in an output?
  6. What would you require before offering the model?

Each rung has a nudge, a model answer, the principle that carries over and the weak answer to avoid. Sign in free to open it.

UP NEXT ON YOUR JOURNEY
Next in this trackExplain differential privacy in plain terms. What do epsilon and delta mean?Next in this trackHow are backdoors implanted through fine-tuning, and why are they hard to find?Next in this trackWhy does exposing confidence scores make extraction and inference attacks easier?
DISCUSSION · 0

No comments yet. Share a correction or a different way to answer it.