promptfoo Explained: LLM Evals and Red Teaming from a YAML File, in CI, and After the OpenAI Deal

promptfoo treats LLM security testing like unit testing: write prompts, providers and assertions in a YAML file, run them as a matrix, and fail the build when results get worse. It also generates red-team tests. Because it lives naturally in CI, it often comes up as the answer to "how do you stop a prompt change from quietly making the product less safe?"

What it is

promptfoo is an open-source command-line tool and library, licensed under MIT and distributed through npm. On 9 March 2026 promptfoo announced that it had agreed to be acquired by OpenAI and said it would remain open source; its README now says it is "now part of OpenAI". It has two main uses:

• Evaluations: run a set of prompts against one or more models and grade the outputs with assertions.
• Red teaming: generate adversarial test cases from plugins and strategies, run them and grade the responses.

In March 2026 promptfoo also open-sourced ModelAudit, a static scanner for model files.

How it works

An evaluation config has prompts (templates with {{variables}}), providers (the models or endpoints), and tests, each with vars and assert entries. Assertion types range from simple string checks (contains, icontains) to javascript functions, cost and latency limits, and llm-rubric, where another model grades the answer.

A red-team config adds a redteam section:

• purpose describes the application, so generated attacks fit it.
• plugins say what to test, such as personal data leakage or indirect prompt injection, or a framework preset such as owasp:llm.
• strategies say how to deliver each test, such as basic, encoded or multi-turn.
• numTests sets how many cases to generate per plugin.

How to use it

Day one: an evaluation you can rerun

npm install -g promptfoo
promptfoo init --example getting-started   # writes promptfooconfig.yaml
promptfoo eval
promptfoo view                              # browse results

Add a few security assertions to your existing tests, such as "the answer never includes the internal discount code" or "the assistant refuses to reveal another customer's order", so security checks run alongside quality checks.

Week one: generate a red-team suite

promptfoo redteam init --no-gui     # or: npx promptfoo@latest redteam setup
promptfoo redteam run               # generate then evaluate
promptfoo redteam report

Write the purpose carefully. A vague purpose produces generic tests; a specific one ("a support assistant that can look up orders by order ID for the signed-in customer only") produces tests about the risks your app actually has. Point the target at a test environment with test data, never production customer data.

Every day: gate pull requests

The promptfoo/promptfoo-action GitHub Action runs evaluations on a pull request and comments with a before-and-after comparison, so reviewers see when a prompt or model change made results worse. For red teaming in CI, npx promptfoo@latest redteam run -o output.json produces machine-readable results, and tagging runs with the commit SHA keeps history traceable. Pin the action to a commit SHA, as with any third-party action.

Where it stops

• Graders are models. llm-rubric assertions and red-team grading rely on a grading model; check a sample of its decisions by hand.
• Generated tests are a sample. Passing them shows resistance to those tests, not to every attack.
• Configuration quality decides results. A wrong target, purpose or plugin selection gives reassuring but irrelevant numbers.
• Ownership change. Some teams will want to watch how vendor neutrality evolves now that promptfoo has agreed to join a model provider; the code remains MIT licensed.

How it comes up in interviews

• "How do you test an LLM feature for security regressions in CI?" Assertions for known-bad outcomes, a generated red-team suite, baselines, and pull request comparisons.
• "What is the weakness of LLM-graded tests?" Grader errors and drift; validate graders with labelled samples.
• "Evals versus red teaming?" Evals check expected behaviour on known cases; red teaming searches for unexpected failures.

Sources

Checked on 13 November 2026: promptfoo GitHub README (promptfoo/promptfoo); promptfoo documentation for getting started, red-team quickstart and configuration, and the GitHub Action integration; promptfoo blog posts "Promptfoo is joining OpenAI" (9 March 2026) and on open-sourcing ModelAudit (3 March 2026); npm registry entry for promptfoo.