Question 1

Evasion versus poisoning attacks: define and contrast them with an example of each.

Accepted Answer

One attacks the model's inputs, the other its upbringing. The phase decides the defence, and interviewers want you to say so.

Question 2

What is membership inference, and why does overfitting raise the risk?

Accepted Answer

The attacker never sees the dataset. They see how much more confident the model is about one record than it should be.

Question 3

What is model extraction, what does the attacker gain, and how do you detect and limit it?

Accepted Answer

The API is the attack surface and every response is a training example. Prevention is unrealistic; slowing, limiting and noticing are not.

Question 4

How can training data be poisoned, and how would you detect it in a pipeline?

Accepted Answer

Detection alone loses: clean-label poisons look correct. The answer that scores builds a pipeline where poison is hard to add, easy to trace and contained when it slips through.

Question 5

What is an adversarial example, and how is it different from an ordinary error?

Accepted Answer

Every model makes mistakes. An adversarial example is a mistake someone chose for it, and that changes how you measure and defend.

Question 6

How do training data extraction and memorisation work in LLMs, and how do you reduce them?

Accepted Answer

Language models are trained to predict text, and sometimes the best prediction is the exact text they saw. Rare, repeated and sensitive strings are the risk.

Question 7

Explain differential privacy in plain terms. What do epsilon and delta mean?

Accepted Answer

The guarantee is about what an observer can learn from your presence in the data, not about hiding the data. Epsilon bounds it; delta is the small print.

Question 8

How are backdoors implanted through fine-tuning, and why are they hard to find?

Accepted Answer

The model passes your evaluation because your evaluation does not contain the trigger. And safety training may teach it to hide better.

Question 9

Why does exposing confidence scores make extraction and inference attacks easier?

Accepted Answer

A label is one bit of information per query. A probability vector is a map of the decision boundary, delivered on request.

Question 10

Accuracy drops 20 percent. How do you tell an attack from drift?

Accepted Answer

Drift is the world changing. An attack is someone changing it on purpose. The evidence that separates them is where and how the errors cluster.

Question 11

Explain FGSM and PGD, and what the perturbation budget means.

Accepted Answer

The two attacks every robustness paper starts from. The scored answer explains the gradient step, the budget as a threat model, and why the iterative attack is the standard for evaluation.

Question 12

How does adversarial training work, and what are its limits?

Accepted Answer

The one defence that has held up in research, at a price. The scored answer explains the min-max objective and names the costs: accuracy, compute, and robustness only to the threat model trained on.

Question 13

What is gradient masking, and why is it a trap when you evaluate defences?

Accepted Answer

A defence that breaks the attacker's tools is not a defence that stops the attacker. The scored answer explains how masking fools gradient attacks, how to spot it, and how adaptive attacks get around it.

Question 14

A team claims their model is now robust to adversarial inputs. How do you evaluate that claim honestly?

Accepted Answer

Most broken defences were evaluated by their authors. The scored answer pins down the threat model, attacks the defence specifically, runs sanity checks, and reports robust accuracy with the conditions attached.

Question 15

If a model only returns accept or reject, how can an attacker still find inputs that evade it?

Accepted Answer

Hiding scores slows attackers down; it does not stop them. The scored answer explains decision-based search and transfer, and turns query patterns into a detection opportunity.

Question 16

How would you red-team a fraud detection model?

Accepted Answer

Fraudsters are the original adaptive adversaries. The scored answer attacks with realistic, feasible changes, probes the whole decision system rather than the model alone, and measures dollars, not just accuracy.

Question 17

What is clean-label poisoning, and why is it harder to catch than label flipping?

Accepted Answer

Every poisoned example looks correctly labelled to a reviewer. The scored answer explains the mechanism, why label audits miss it, and the defences that work at the pipeline level.

Question 18

How much poisoned data does an attacker need, and why does that matter at web scale?

Accepted Answer

The intuition that a huge dataset dilutes poison turns out to be wrong for some attacks. The scored answer cites the evidence, explains why, and moves the defence to data provenance and testing.

Question 19

Model inversion, model extraction and attribute inference: what is the difference?

Accepted Answer

Three attacks that all start with queries and end with something the owner did not intend to share. The scored answer separates them by what is stolen and maps each to its defence.

Question 20

Is anonymising training data enough to stop privacy attacks on a model?

Accepted Answer

Removing names removes the obvious identifier and leaves the rest. The scored answer explains re-identification, why models amplify it, and what actually bounds the risk.

Question 21

Explain DP-SGD, its privacy and utility trade-off, and when differential privacy is the wrong tool.

Accepted Answer

DP gives a real guarantee with a real cost. The scored answer explains clipping, noise and accounting, then compares DP with cheaper controls for a concrete dataset.

Question 22

A deployed model may be leaking its training data. How do you respond?

Accepted Answer

The leak is in the weights, so the playbook differs from a database breach. The scored answer confirms, contains at the output, scopes from data lineage, and decides between filtering, retraining and withdrawal.

Question 23

What can watermarking of model outputs or weights prove, and what can it not?

Accepted Answer

A watermark is evidence of origin under conditions, not a detector of all AI text. The scored answer explains how it works, how it fails, and what claims it can support.

Question 24

Why is membership inference hard even to define for LLMs trained on internet-scale data?

Accepted Answer

A research-depth question. The scored answer explains why 'was this in training' blurs at web scale, why many published attacks measured distribution shift instead, and what questions remain answerable.

Question 25

Why can adversarially robust models leak more membership information about their training data?

Accepted Answer

Two security goals that pull against each other. The scored answer explains the mechanism, the evidence, and how to make the trade-off explicit in a design.

Question 26

You suspect a fine-tuning dataset was poisoned. How do you investigate, prove it, and decide what to do with the model?

Accepted Answer

'Clean the dataset' is not an investigation. The answer freezes evidence, traces what changed and who could change it, tests causality by retraining, and treats the pipeline as possibly compromised too.

Question 27

What is transferability of adversarial examples, and why doesn't hiding a model behind an API stop adversarial attacks?

Accepted Answer

An attacker who cannot see your model can attack one they can see, and much of the damage carries over. That single property turns 'the weights are secret' into a weak defence.

Question 28

How do adversarial images attack vision-language models, how is that different from text in images, and does preprocessing help?

Accepted Answer

Two different attacks hide in 'malicious images'. One writes instructions a model can read; the other optimises pixels to push the model's internal representation. They need different tests and different defences.

Question 29

How would you test whether a model update increased memorisation, and does a leak that needs thousands of queries matter?

Accepted Answer

Generic leakage benchmarks will not tell you whether your customers' records got easier to extract. Planted canaries, matched controls and an attack-economics view of severity will.

Question 30

How would you red-team a content moderation classifier for evasion, and report results that are useful to the team that owns it?

Accepted Answer

Evasion testing is a search problem with a budget. Start from real harmful content, transform it while keeping its meaning, adapt to what gets through, and report false negatives and false positives by category.