Skip to content
AI Security

Interview question · AI Red Teaming, Evaluation & Response

When do you red-team manually and when with automated tools?

Choose people and automation by the evidence the task needs. Validate candidates, calibrate scoring and separate exploratory search from a defensible estimate.

mediumNewUpdated Nov 2026

Choose people and automation by the evidence the task needs. Validate candidates, calibrate scoring and separate exploratory search from a defensible estimate.

Make your first attempt

What would you say before reading the answer?

  1. 01 State the assumptions your answer depends on.
  2. 02 Explain the mechanism and choose a control.
  3. 03 Name a condition that would change your decision.

The worked answer includes an approach, scoring criteria and follow-up questions. Compare the reasoning with your own.

signing in unlocks more free answers per topic

What if · the follow-ups in this answer

The answer works through 4 follow-ups like an interviewer would

An automated benefits-assistant search flags a restricted employee field in a draft. Its scorer cannot see whether the draft is staff-only or employee-visible.

  1. What if a human did not anticipate this candidate?
  2. What if a domain reviewer and model judge disagree?
  3. What if the next automated run has more successes after doubling its budget?
  4. How would you improve this assessment process?

Each rung has a nudge, a model answer, the principle that carries over and the weak answer to avoid. Sign in free to open it.

UP NEXT ON YOUR JOURNEY
Next in this trackWhat AI-specific telemetry should you log?Next in this trackWhy do you still need human review with automated judges, and how do you validate an LLM judge?Next in this trackHow do you put security regression tests for prompts and guardrails into CI?
DISCUSSION · 0

No comments yet. Share a correction or a different way to answer it.