Skip to content

Interview question · AI Red Teaming, Evaluation & Response

Why can a single evaluation number mislead you about an AI system's security?

A falling aggregate can hide worse observed results in every input slice. Learn to inspect mixtures, missing outcomes, scorer changes and the release requirement.

hardNewUpdated Nov 2026

A falling aggregate can hide worse observed results in every input slice. Learn to inspect mixtures, missing outcomes, scorer changes and the release requirement.

Make your first attempt

What would you say before reading the answer?

  1. 01 State the assumptions your answer depends on.
  2. 02 Explain the mechanism and choose a control.
  3. 03 Name a condition that would change your decision.

The worked answer includes an approach, scoring criteria and follow-up questions. Compare the reasoning with your own.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 529 remaining answers · $25

Every worked answer in all 17 topics, every lesson in every course, the full concept curriculum, the complete practice-test bank, and the downloadable security field guides.

6 months · One payment · No auto-renewal

What if · the follow-ups in this answer

The answer works through 4 follow-ups like an interviewer would

Run A of a document-assistant assessment fails twenty of a hundred document cases and one of a hundred message cases. Run B fails three of ten document cases and nine of three hundred message cases.

  1. What if the release note advertises a drop from 10.5% to 3.9%?
  2. What if the document slice has only ten cases in the second run?
  3. What if a private holdout performs worse than the shared suite?
  4. How should this result inform a release decision?

Each rung has a nudge, a model answer, the principle that carries over and the weak answer to avoid. Premium opens every answer and ladder.

UP NEXT ON YOUR JOURNEY
DISCUSSION · 0

No comments yet. Share a correction or a different way to answer it.