Hybrid retrieval and per-user permissions in the index: how not to leak data in RAG

Most security incidents in RAG systems don't come from sophisticated prompt injection. They come from something more mundane: the search index returns a document the user wasn't allowed to see at the source. The root cause is almost always the same — permissions were treated as an implementation detail, not as part of the retrieval design.

This article covers how to build hybrid retrieval (lexical + vector) with correct access control: where to apply the filter, how to sync revocations, how to isolate tenants, and how to test that none of it is broken.

Why hybrid, not just vector

Pure vector search fails in two common scenarios: exact terms (product codes, IDs, acronyms) and negations or structured filters. Lexical search (BM25 or equivalent) covers exactly those cases.

The hybrid architecture runs both in parallel and fuses the results:

User query
   │
   ├─→ Lexical search (BM25)       ──┐
   │                                  ├─→ Fusion (RRF or weighted score) → Final top-K
   └─→ Vector search (embeddings)  ──┘

Reciprocal Rank Fusion (RRF) is the more robust choice when the two lists have incomparable score scales: instead of normalizing scores, you sum 1 / (k + rank) for each document across both lists and re-sort by the total. This avoids having to recalibrate weights every time one of the engines changes version.

A simpler — and more fragile — alternative is normalizing scores (min-max) and combining them with a fixed weight (e.g., 0.6 vector + 0.4 lexical). It works, but it requires recalibration whenever the score distribution shifts.

Where to apply the ACL: pre-filter vs. post-filter

This is where most RAG systems get it wrong.

Post-filtering — search first, filter by permission afterward — is the more common approach because it's simpler to implement. The problem: if the user has no access to any of the returned top-K documents, the final answer ends up empty or poor, even though permitted documents exist further down the ranking. Worse, depending on the implementation, counts and aggregations can leak the existence of documents (the AI mentions "there are 3 documents about X" even though the user can't see any of them).

Pre-filtering — restricting the search space to the user's permissions before ranking — is the correct approach for sensitive data. The ACL filter becomes part of the query to the search engine (a metadata filter), not a later in-memory step:

Query + user ACL (groups, tenant, roles)
   │
   ▼
Search engine filters by ACL BEFORE ranking
   │
   ▼
Top-K is already 100% within what the user can see

Most modern vector databases (Pinecone, Weaviate, Qdrant, pgvector with RLS) support native metadata filtering fast enough that it won't become a bottleneck, as long as the ACL field is indexed.

Modeling ACL inherited from the source

The index shouldn't reinvent permissions — it should mirror the ones from the source (SharePoint, Google Drive, Confluence, a database). Every indexed chunk carries access-control metadata:

At query time, resolve the user's identity (groups, roles, tenant) and compose the filter: tenant_id = X AND (acl_groups INTERSECTS user_groups OR acl_users CONTAINS user_id).

Revocation and permission sync

Permissions inherited from the source are only trustworthy if the index is updated when the source changes. Two strategies, best combined:

1. 1.Event-driven sync: the source emits a webhook or event (e.g., a user removed from a group in SharePoint) and the index reprocesses the affected documents near real time.
2. 2.Periodic reconciliation: a job runs on a fixed cadence (e.g., every 15–30 minutes for sensitive data, hourly for the rest) and recomputes ACLs from the source, correcting drift that events missed.

Treat periodic reconciliation as a mandatory safety net, not an optional extra — event systems fail silently, and reconciliation is what catches missed revocations before they turn into an incident.

Multi-tenancy

Multi-tenancy is permission at maximum scale: an isolation bug between tenants is the worst kind of leak, because it crosses organizational boundaries.

• Prefer physical or namespace isolation (an index or collection per tenant) when volume allows it — it eliminates an entire class of filter bugs.
• When the index is shared, tenant_id must be a mandatory filter, enforced at the infrastructure level (database policy, not just application code) — so an application bug can't turn into a cross-customer leak.
• Never rely on a tenant filter that can be accidentally omitted in a specific query; use row-level security or an equivalent that rejects queries without the filter.

Automated leakage tests

ACL without automated testing is ACL that will eventually leak; it's just a matter of time. The minimum test suite:

• Tenant isolation test: a user from tenant A should never see, in any query, a document from tenant B — run against a fixture dataset covering multiple tenants.
• Revocation test: remove a user's permission and verify that a query immediately after no longer returns the document (within the accepted sync window).
• Aggregation leakage test: verify that counts, summaries, and AI suggestions don't reveal the existence of documents outside the user's scope.
• Privilege escalation test: simulate a user with no group at all and confirm that zero restricted documents come back.

Run this suite in CI, against a test index with synthetic data representing multiple tenants and permission levels — not just manually before a major release.

What to do on Monday

• Audit whether your RAG pipeline applies ACL as a pre-filter or a post-filter in search, and migrate to pre-filtering if it isn't already the case.
• Check whether periodic permission reconciliation exists, not just event-driven sync, and define the cadence for sensitive data.
• Write the first automated tenant-isolation test and run it in CI before the next deploy.
• If the index is shared across tenants, confirm the tenant_id filter lives at the infrastructure level, not just in application code.

Further reading

Executive track:

• RAG is not magic: the real cost of giving AI enterprise context — the business view of this same topic.