TL;DR: An ordinary error is a mistake on naturally occurring data, measured by average accuracy on a test set. An adversarial example is an input deliberately optimised by an attacker, often with a perturbation people cannot perceive, to cause a specific error with high confidence. Ordinary errors are reduced by better data and models; adversarial errors are a worst-case property that average accuracy does not measure, so robustness must be evaluated against adaptive attackers within a stated threat model (perturbation budget, attacker knowledge).
How to approach it
Contrast the two along four axes: who chooses the input, what is measured, how often it occurs, and how you defend. Use one concrete example, then mention LLM analogues.
A strong answer
Ordinary error
Natural data
measured on a test set
Occasional mistake
no intent
Better data reduces it
Outcome, safe: Average-case problem
Accuracy metrics capture it
Adversarial example
Deliberate optimisation
imperceptible change
Specific, confident error
chosen by the attacker
Survives high accuracy
Outcome, unsafe: Worst-case problem
Needs robustness evaluation
| Ordinary error | Adversarial example | |
|---|---|---|
| Input source | Nature, users, data distribution | An attacker optimising against the model |
| Frequency | Some rate on typical data | Near-certain once the attacker searches |
| Measured by | Accuracy on held-out data | Accuracy under attack, within a stated threat model |
| Confidence | Often low near boundaries | Often high: the attacker pushes past the boundary |
| Fix | Better data, architecture, training | Adversarial training, certified robustness, system design |
Example. An image classifier labels a few blurry stop signs as speed-limit signs: ordinary errors, more data helps. Researchers add a few carefully designed stickers to a clear stop sign so the classifier reads it as a speed-limit sign from many angles (Eykholt et al., 2018): an adversarial example, and more ordinary data does not fix it.
Why they exist. Models rely on statistical features that humans ignore, and in high-dimensional inputs many tiny coordinated changes can add up to a large change in the model's output (Goodfellow et al., 2014).
Defending. Adversarial training with strong attacks such as PGD (Madry et al., 2017) improves robustness within the trained threat model; certified methods give guarantees in small regions; system design (multiple signals, human review for high-stakes decisions, query limits) limits consequences. Evaluate against adaptive attacks: many published defences were later broken (Athalye et al., 2018).
In LLMs. Optimised adversarial suffixes that cause aligned models to comply (Zou et al., 2023) and jailbreaks that evade classifiers are the text equivalents: inputs chosen to land on the wrong side of a learned boundary.
What interviewers probe next
- "Does robustness cost accuracy?" Adversarial training typically costs some clean accuracy; the trade-off depends on the task and budget.
- "What is a threat model for robustness?" The allowed perturbation (type and size), attacker knowledge and query budget; robustness claims are meaningless without one.
- "Are adversarial examples a real-world threat or a lab curiosity?" Real where attackers control inputs and benefit from errors: spam, malware, fraud, content moderation, safety filters.
What if?
What if · interview follow-ups
Hold your answer as the constraints change
A product manager sees a demo where a stop sign with a few stickers is classified as a speed limit sign, and asks whether this is just a bug that more training data will fix.
Start here
How is this different from an ordinary error?
Did the stickers occur naturally?
An ordinary error is a mistake on naturally occurring data, measured by average accuracy. An adversarial example is deliberately optimised by an attacker to cause a specific error, often with high confidence and changes people barely notice. It is a worst-case property that average accuracy does not measure.
Carry it forwardAdversarial errors are optimised, not accidental.
Weak answerTreating adversarial examples as rare edge cases.
Constraint changes
Will more training data fix it?
Does the attacker pick inputs from your data distribution?
More natural data reduces ordinary errors but does not remove adversarial ones, because the attacker searches for inputs the model handles badly, wherever they are. Defences such as adversarial training help within a defined threat model but cost accuracy and compute and do not generalise to every kind of change.
Carry it forwardMore data reduces average errors, not worst-case ones.
Weak answerPromising the next retrain will fix it.
Constraint changes
What is a threat model for adversarial examples?
What changes are allowed in the attack?
A statement of what the attacker can change and by how much (for example, a small per-pixel change, or physical stickers on a sign), what they know about the model, and what outcome they want. Resistance claims only mean something against a stated threat model.
Carry it forwardClaims need a threat model: budget, knowledge and goal.
Weak answerAsking whether a model is 'secure against adversarial examples' in general.
Constraint changes
What if the product is a document classifier, not a camera?
What can an attacker change in a document?
The same idea applies with different budgets: changes to words, formatting or file structure that keep meaning but change the label. Which changes are realistic depends on what the attacker controls. Define them per product and test with adaptive attackers.
Carry it forwardEach domain has its own feasible perturbations.
Weak answerCopying image threat models to text.
Constraint changes
What should the product do about it?
What else could catch a fooled classifier?
Avoid making a single model's decision the only safeguard where errors are dangerous: combine it with other sensors or signals, check plausibility, require human review for high-impact outcomes, and evaluate against adaptive attacks within a stated threat model. Design the system so one fooled model is not a disaster.
Carry it forwardDesign systems so one fooled model is not enough.
Weak answerRelying on a single classifier for safety-critical decisions.
Now defend it
How would you explain it to the product manager?
An adversarial example is an input deliberately crafted to cause a specific error, so it is a worst-case property that average accuracy does not show. More data does not fix it. Define a threat model, evaluate against adaptive attackers within it, use defences such as adversarial training where justified, and design the product so other signals and review catch a fooled model.
How it's scored
Weak: "A wrong prediction on a weird input."
Hire: Distinguishes chosen versus natural inputs and average versus worst-case evaluation.
Strong Hire: Explains why they exist, the threat-model requirement for robustness claims, adaptive evaluation, and LLM analogues.
Common mistakes
- Reporting accuracy as evidence of robustness.
- Robustness claims without a perturbation budget.
- Trusting defences evaluated only on non-adaptive attacks.