Skip to content

Interview question · AI Red Teaming, Evaluation & Response

How would you put an LLM red-team scanner such as garak, PyRIT or promptfoo into CI without it blocking every release?

Scanners produce rates, and rates are noisy. A strong answer picks tests from the threat model, fixes the sampling, validates the grader, gates on meaningful change and keeps manual red teaming for what automation cannot find.

hardNewPreparation tags: Microsoft, NVIDIA, OpenAIUpdated Nov 2026

Scanners produce rates, and rates are noisy. A strong answer picks tests from the threat model, fixes the sampling, validates the grader, gates on meaningful change and keeps manual red teaming for what automation cannot find.

Make your first attempt

What would you say before reading the answer?

  1. 01 State the assumptions your answer depends on.
  2. 02 Explain the mechanism and choose a control.
  3. 03 Name a condition that would change your decision.

The worked answer includes an approach, scoring criteria and follow-up questions. Compare the reasoning with your own.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 533 remaining answers · $25

Every worked answer in all 17 topics, every lesson in every course, the full concept curriculum, the complete practice-test bank, and the downloadable security field guides.

6 months · One payment · No auto-renewal

What if · the follow-ups in this answer

The answer works through 4 follow-ups like an interviewer would

Your team added promptfoo red-team tests to every pull request. After a month, developers say the check is flaky, and a recent change that made the assistant leak internal tool names passed the gate.

  1. Why might the gate be flaky?
  2. Why did the tool-name leak pass?
  3. How do you restore trust in the gate?
  4. How do you summarise putting an LLM red-team scanner into CI?

Each rung has a nudge, a model answer, the principle that carries over and the weak answer to avoid. Premium opens every answer and ladder.

UP NEXT ON YOUR JOURNEY
DISCUSSION · 0

No comments yet. Share a correction or a different way to answer it.