RAG Security

TL;DR: A RAG system copies documents into the model's context on demand, so it must enforce the same access rules as the systems it copied from. Copy permissions into chunk metadata at ingestion, filter retrieval on the asking user's identity before ranking, isolate tenants, keep permissions and deletions in sync, and never rely on the model to withhold what it was given. Then treat every retrieved chunk as untrusted input (its author can inject), and handle the answer like any other model output.

What you should be able to do

Trace one document from its source through chunks, embeddings, retrieval, reranking and a generated answer. Identify the last trusted permission check before each recipient sees its text. Explain how a permission change invalidates an already cached answer. If the pipeline is unfamiliar, first read how LLM applications work.

Three properties a secure RAG system needs

1. Retrieval enforces authorization. The model will use whatever is in its context. If a chunk the user may not read reaches the prompt, the only remaining defence is the model's discretion, which is not a defence. Enforce the decision inside the trusted retrieval service before any disallowed text reaches the caller, model or an unauthorized downstream processor.

2. Retrieved content is untrusted. Any document someone can write is a channel for indirect prompt injection and for misinformation. Internal does not mean trusted.

3. The answer is handled safely. It may quote retrieved text verbatim, include links, or be rendered as markdown; standard output handling applies.

Getting authorization right

At ingestion: record, for every chunk, the source document ID, tenant, and the principals (users, groups) allowed to read it. Strip or restrict documents whose permissions cannot be represented.

At query time: filter by tenant and by the user's principals inside the search, so ranking only considers allowed chunks. For high-sensitivity data, re-check permissions against the source system for the final set of chunks.

Over time: propagate permission changes and deletions with a defined maximum lag, and test revocation. Cache scope must reflect the caller's actual access, not just the tenant. Include an authorization version or invalidate affected entries when access changes, and authorize the source documents again before serving a sensitive cached answer.

Isolation: enforce tenant scope in a mandatory retrieval boundary. Engine-level policies, separate credentials or separate indexes can reduce the chance that an omitted application filter exposes another tenant. Separate indexes on shared infrastructure are logical separation, not automatically separate compute or administrative access.

Worked example: post-filtering and retrieval quality

A corpus holds 2,000,000 chunks. Priya, through three groups, may read 40,000 of them: 2%.

• Post-filter. Retrieve the top 10 from the whole corpus, then drop what Priya may not see. If relevance is unrelated to permission, the expected number of allowed results is 10 × 0.02 = 0.2 per query: most answers come back empty. Increasing the candidate count to 500 gives an expectation of 10 allowed results under the same simplifying assumption, at extra cost. This is illustrative arithmetic, not a measured production result. Filtering inside a trusted retrieval service can preserve confidentiality if disallowed chunks never reach the model, an unauthorized reranker, logs or the caller; the concern here is recall and the pressure to weaken policy.
• Pre-filter in the index. Pass Priya's principals (her user ID and three groups) and the tenant as a filter inside the approximate nearest-neighbour search, so the top 10 are drawn only from documents she may read. Test retrieval quality under a 2% filter on your engine, because filtered approximate search behaves differently across vector databases. Azure AI Search documents the filter pattern and explicitly distinguishes a filter string from authenticating the caller.
• Chunking. The permission on the parent PDF must be copied to every chunk. Microsoft's Azure AI Search documentation warns that without this projection, chunk-level references are not filtered.
• Revocation. Priya leaves group g2 at 09:00. If the indexer that syncs permissions runs every six hours, she can retrieve g2's documents until 15:00. Shorten the sync for sensitive sources, or, for the most sensitive tier, re-check final results against the source system's ACL at answer time.
• Caches. Include tenant, effective access scope, authorization version and relevant corpus/model versions. A group-membership hash alone does not reflect a changed document ACL. Keep source IDs with cached answers so a cache hit can be checked against current permission and deletion state.

Isolation options by engine

Vector stores offer different isolation primitives, and the choice trades isolation strength against cost. Weaviate's multi-tenancy stores each tenant on a separate shard and requires the tenant name on every operation. PostgreSQL with pgvector can use row-level security, with the caveat from the broken access control lesson: superusers, roles with BYPASSRLS and, unless FORCE ROW LEVEL SECURITY is set, the table owner bypass it, so a retrieval service connecting as the owner silently ignores every policy. Azure AI Search offers string-match security filters that the application passes, and newer modes that take the user's Entra token and trim results using synchronised permission metadata. Choose the boundary that matches the threat: an engine policy, an unavoidable retrieval service, separate credentials, or dedicated infrastructure. Test that no caller can skip tenant and document checks, including through administrative and batch interfaces.

Treating retrieved text as untrusted

• Know who can write each source. A wiki every employee can edit is an injection channel for any compromised account.
• Mark retrieved text as data in the prompt and use a model trained with an instruction hierarchy. This lowers, not removes, injection risk.
• Limit what a RAG answer can trigger. A bot without tools still produces text that can disclose confidential context or mislead a user. Keep unauthorized documents out of its context, constrain rendering and cite sources; removing tools reduces available actions but does not remove disclosure risk.
• Watch for poisoning: documents designed to rank highly for specific queries and carry instructions or false claims. Review changes to high-traffic sources and monitor for documents that suddenly dominate retrieval.

Poisoning: five documents in millions

PoisonedRAG (Zou and colleagues, USENIX Security 2025) showed how little an attacker needs. Five crafted texts per target question achieved about 90% attack success against knowledge bases of 2.7 to 8.8 million passages. An attack must satisfy two conditions, and they are worth testing separately: the poisoned text must be retrieved into the top results for the target question, and the model must generate the attacker's answer from it. The defences the authors tested did not hold: paraphrasing the query lowered success only modestly, perplexity-based detection produced too many false positives, removing duplicates did nothing because the poisoned texts differed, and retrieving many more passages still left 41% to 43% success. The controls that matter are about who can write to the corpus: write permissions on sources, provenance and trust weighting kept separate from similarity scores, monitoring for documents that suddenly dominate retrieval for high-value queries, and canary queries on sensitive topics.

Embeddings are sensitive data

Vectors look like opaque numbers, so teams store them with less care than the text. Morris and colleagues ("Text Embeddings Reveal (Almost) As Much As Text", EMNLP 2023) recovered 92% of 32-token inputs exactly from their embeddings with an iterative correct-and-re-embed method, and recovered full names from clinical notes. The 92% result applies to the paper's tested models and short inputs, not every embedding system. It is still enough to reject the claim that vectors are anonymized: encrypt it, restrict access to it, include it in data-deletion workflows and breach assessments, and do not share embeddings with third parties you would not share the text with.

Common RAG failures

Exercise: revoke access without changing the question

Use a local fixture with two tenants and three short documents: one public, one private to each tenant. Give the retrieval service an authenticated principal from trusted server state. The client supplies the search question, never its own tenant permission filter.

If revocation propagates asynchronously, name the maximum delay your design promises and test that delay. Immediate revocation requires a current authorization check or an equivalent invalidation mechanism. A visible citation alone does not prove that the caller may read its source.

This exercise tests confidentiality and permission freshness. A separate evaluation is needed for whether the answer is accurate and supported by its permitted sources. Run fixtures only in systems you own or are authorised to test.

Misconceptions interviewers listen for

• "The model will respect an instruction not to reveal restricted documents." It has already been given the text; authorisation must happen before retrieval.
• "Embeddings are anonymised." Research has recovered sensitive text in tested settings; recovery depends on the model, input and attacker access.
• "Deduplication stops poisoning." Poisoned texts can all differ.
• "Separate indexes per tenant remove the need for authorisation." Users within a tenant still have different permissions.
• "Permissions are checked at ingestion." They change after ingestion; sync lag is an exposure window.

Primary references

References for the mechanisms discussed above, accessed 6 November 2026:

• OWASP authorization: validate permissions on every request.
• PoisonedRAG, USENIX Security 2025: experimental setting and results.
• Text Embeddings Reveal (Almost) As Much As Text, EMNLP 2023.

How this shows up in interviews

"Secure a RAG application end to end" and "enforce document-level permissions in RAG" are two of the most common AI application security questions. Lead with authorization at retrieval, because a permission error can expose documents without any model manipulation, then cover untrusted content, output handling, and operations (sync, deletion, caching, monitoring).

Say it in 30 seconds: "RAG copies data into the model's context, so retrieval has to enforce the same permissions as the source: copy ACLs and tenant into chunk metadata, filter on the asking user's identity before ranking, isolate tenants, and sync permission changes and deletions. Never rely on the model to withhold what it was given. Then treat retrieved text as untrusted because its authors can inject, and handle the answer like any model output."