How to Use Metadata Filtering in RAG Retrieval

// In plain English

In a RAG system, the retriever finds chunks whose meaning is closest to the question. That is powerful, but meaning is not the only thing that matters. The most relevant-sounding paragraph might belong to a different customer, an expired policy, or a document the current user is not allowed to see. Pure similarity search has no idea about any of that.

Metadata filtering fixes this by attaching structured labels to every chunk — things like tenant_id, date, source, language, or access_level — and then telling the search to only consider chunks that match a condition before (or after) ranking by similarity. Similarity decides which chunk is most relevant; the filter decides which chunks are even allowed to compete.

Think of a library with a brilliant librarian who finds the perfect book on any topic. Metadata filtering is the rule you give them first: only look in the children's section, or only books published after 2020, or only books this member is permitted to borrow. The librarian's skill at matching topics is unchanged. You have simply fenced off the shelves they are allowed to search.

// Why it matters

Once a RAG app leaves the demo stage and serves real users with real data, similarity alone starts returning answers that are relevant but wrong. Metadata filters are how you keep retrieval correct. Four cases come up again and again.

• Multi-tenant isolation. One vector store holds documents for many customers. Without a tenant_id filter, Customer A's question can pull back a chunk written for Customer B — a data leak, not a bad answer. This is the single most important filter in any B2B RAG app.
• Freshness and validity. Prices, policies, and schedules change. A date or valid_until filter keeps the model from grounding its answer in a document that expired last quarter, even if that old document is the closest semantic match.
• Source and document-type scoping. "Answer only from the official handbook, not from draft notes" or "search support tickets, not marketing pages." A source or doc_type filter narrows retrieval to the right kind of content for the question.
• Access control. Different roles see different documents. An access_level or allowed_roles filter ensures a contractor's query never retrieves an HR file or a confidential contract. Permissions enforced at retrieval time are far safer than hoping the model declines to repeat what it was handed.

The thread through all four: the filter encodes a rule the embedding model can never learn on its own. An embedding captures topic and tone, not ownership, recency, or permission. Those are facts about the document, and the only place to enforce them is the metadata you attach and query. Skip them and your RAG system is one unlucky similarity score away from a leak or a stale answer.

// How it works

Metadata filtering touches two points in the pipeline. At ingestion, every chunk is stored with a small bag of structured fields alongside its vector. At query time, the request carries both the embedded question and a filter expression; the vector store applies the filter and ranks by similarity together.

Step 1 — attach metadata at ingestion

When you chunk and embed a document, you also record what you know about it. Keep the fields small, structured, and filterable — exact values (strings, numbers, dates, booleans), not long prose. Every vector store stores this metadata next to the vector.

store.add(
    id="doc42-chunk3",
    vector=embed(chunk_text),
    metadata={
        "tenant_id": "acme-corp",
        "source": "handbook",
        "doc_type": "policy",
        "access_level": "employee",
        "published": "2026-03-01",   # ISO date, easy to range-filter
        "lang": "en",
    },
)

Step 2 — filter at query time

The request combines the embedded question with a filter that the current user and context determine. The store returns the top matches that also satisfy the filter — never the global nearest neighbors.

results = store.query(
    vector=embed(user_question),
    top_k=5,
    filter={
        "tenant_id": current_user.tenant_id,      # isolation — always on
        "access_level": {"$in": current_user.roles},  # permissions
        "published": {"$gte": "2026-01-01"},      # freshness
        "source": "handbook",                     # scope
    },
)
# Only chunks matching ALL conditions are ranked by similarity.

The filter syntax varies by store but the operators are familiar: equality, $in (one of a set), range comparisons ($gte, $lte) for dates and numbers, and boolean $and / $or to combine conditions. The hard part is not the syntax — it is deciding where in the search the filter runs, which is the next section.

// Pre-filter vs post-filter

There are two moments a filter can run relative to the similarity search, and the choice has real consequences for both correctness and recall.

• Pre-filtering restricts the candidate set first, then ranks only the survivors by similarity. The result is always the true top-k within the allowed subset. This is what you want for correctness — a tenant or permission filter must never be bypassed.
• Post-filtering runs the similarity search over everything first, takes the top-k, and then discards the rows that fail the filter. It is simpler for the index, but it can return fewer than k results — or even zero — because the matches it found might all get thrown away.

There is a subtle recall trade-off even with pre-filtering. Many vector indexes are approximate — they explore a neighborhood of the graph rather than every vector. When a filter is very selective (say it keeps only 0.1% of chunks), the approximate walk may not find enough matching neighbors and recall drops. The fixes: raise the search effort (ef_search / number of probes) for selective filters, or for tiny tenants fall back to an exact scan, which is fast when the allowed set is small.

// A worked example: scoping a support bot

Imagine a support assistant serving many companies. Each company has public docs, internal runbooks, and per-plan articles. A logged-in support agent at Acme asks: "What's the SLA for priority incidents on the Enterprise plan?" Here is how the filter is built from context, not from the question text.

Notice that none of these values come from the user's sentence — they come from the authenticated session and account. That is the safe pattern: derive filters from trusted context, never from the user's free text. If you let the model or the user choose the tenant_id, you have handed away the very isolation the filter exists to provide.

def retrieve(question, session):
    flt = {
        "tenant_id": session.tenant_id,        # trusted, server-side
        "plan": session.account.plan,
        "access_level": {"$in": session.roles},
        "published": {"$gte": "2026-01-01"},
        "lang": session.lang,
    }
    return store.query(vector=embed(question), top_k=5, filter=flt)

With this filter in place, the embedding model is free to do what it is good at — matching the meaning of "SLA for priority incidents" — but it can only ever rank documents that are Acme's, Enterprise-plan, agent-visible, current, and English. Relevance and correctness are now handled by two separate, dependable mechanisms.

// Common pitfalls and practical tips

• Over-filtering to zero. A filter that is too strict (exact timestamp instead of a range, an AND of five rare conditions) can leave nothing to retrieve. Log how many candidates survive the filter; if it is routinely near zero, loosen it or widen the date range.
• Free-text values as metadata. Filtering on a messy field like author = "J. Smith " (trailing space, inconsistent casing) silently matches nothing. Normalize values at ingestion — lowercase, trim, use enums or IDs, not human-typed strings.
• Dates as strings that don't sort. Store dates in a sortable form (ISO 2026-03-01 or a Unix timestamp) so range filters work. "March 1" cannot be range-compared.
• Relying on post-filter for security. As covered above, an isolation filter applied after ranking can leak when combined with caching or fallbacks. Make tenant and permission filters a hard pre-filter that the query path cannot skip.
• Forgetting filters break hybrid search. If you run hybrid search (vector + keyword) or reranking, the same metadata filter must apply to every retrieval path. A keyword branch that ignores the tenant filter reopens the leak you closed on the vector side.

// Going deeper

The basic pattern — attach fields, pre-filter by trusted context — carries most RAG apps a long way. A few directions matter once you scale or harden the system.

Partitioning vs filtering. For very large multi-tenant systems, a per-tenant filter on one giant index is not the only option. Some stores let you put each tenant in its own namespace, collection, or partition, so the search never touches other tenants' vectors at all. That gives stronger isolation and faster queries than a runtime filter, at the cost of more objects to manage. A common rule: filter for low-cardinality scopes (a handful of doc_type values), partition for high-cardinality isolation (thousands of tenants).

Row-level security and per-document ACLs. Real permission models are messier than one access_level field — a document might be visible to a specific list of users or groups. Storing an allowed_groups array in metadata and filtering with an $in against the user's group memberships is the usual approach. For strict compliance, push the same rule into the database's own row-level security so it holds even if the application code forgets the filter.

Selective-filter recall, revisited. With approximate indexes (HNSW and friends), the interaction between a strict filter and the graph walk is the deepest gotcha here. If a tenant has only 50 documents inside a 50-million-vector index, the approximate search may wander through millions of disallowed neighbors and surface too few of the 50 that matter. Strategies include raising search effort for small allowed sets, maintaining per-tenant sub-indexes, or detecting tiny result sets and falling back to a brute-force scan over just that tenant's vectors.

Where to go next. Metadata filtering composes with everything else in retrieval: it scopes the candidate pool that hybrid search draws from and that reranking then re-scores. Understanding the full retrieval pipeline and the retriever's role makes it clear where each filter belongs. The durable lesson: keep relevance (embeddings) and eligibility (metadata) as two separate concerns, and never let the model's notion of similarity decide which documents a user is allowed to see.