ColBERT

// Overview

ColBERT is a retrieval model from Stanford that scores how well a passage matches a query using token-level vectors instead of a single embedding per document. It encodes each passage into a matrix of token embeddings, encodes the query the same way at search time, and compares them with a MaxSim operator. This fine-grained, "late interaction" matching lets it rank results more accurately than single-vector models while still scaling to large corpora.

It is aimed at developers and researchers building search and retrieval-augmented generation (RAG) pipelines who need ranking quality beyond a plain vector similarity lookup. You index a collection once, then issue queries to retrieve the top-k passages for each one. ColBERTv2 ships a checkpoint trained on the MS MARCO passage ranking task that you can use directly.

Within the rerankers and hybrid search space, ColBERT sits between dense bi-encoders and slower cross-encoders: it keeps token-level detail like a cross-encoder but precomputes passage representations so retrieval stays fast. If you prefer a higher-level wrapper, the README points to the RAGatouille library, which builds on ColBERT.

// What it does

• Late-interaction scoring: encodes passages and queries into token-level embedding matrices and ranks with a MaxSim operator
• Ships a pre-trained ColBERTv2 checkpoint trained on MS MARCO passage ranking
• Python API with Indexer and Searcher classes built around Run, RunConfig, and ColBERTConfig
• PLAID-based indexing precomputes passage representations on disk for fast top-k retrieval
• Works with a simple tab-separated (TSV) file format for queries, passages, and ranked lists
• Optional training of your own ColBERT model and support for additional Hugging Face models

// Getting started

Install ColBERT, then index a collection and search it with the Python API. A GPU is required for training and indexing.

pip install colbert-ai[torch,faiss-gpu]

from colbert.infra import Run, RunConfig, ColBERTConfig
from colbert import Indexer

if __name__=='__main__':
    with Run().context(RunConfig(nranks=1, experiment="msmarco")):
        config = ColBERTConfig(
            nbits=2,
            root="/path/to/experiments",
        )
        indexer = Indexer(checkpoint="/path/to/checkpoint", config=config)
        indexer.index(name="msmarco.nbits=2", collection="/path/to/MSMARCO/collection.tsv")

from colbert.data import Queries
from colbert.infra import Run, RunConfig, ColBERTConfig
from colbert import Searcher

if __name__=='__main__':
    with Run().context(RunConfig(nranks=1, experiment="msmarco")):
        config = ColBERTConfig(
            root="/path/to/experiments",
        )
        searcher = Searcher(index="msmarco.nbits=2", config=config)
        queries = Queries("/path/to/MSMARCO/queries.dev.small.tsv")
        ranking = searcher.search_all(queries, k=100)
        ranking.save("msmarco.nbits=2.ranking.tsv")

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Build the retrieval stage of a RAG pipeline where ranking quality matters more than a plain vector lookup
• Rerank or retrieve passages over a large corpus when single-vector embeddings miss relevant results
• Run passage search experiments on MS MARCO or your own TSV collection
• Train or fine-tune a domain-specific late-interaction retriever on your own data

// How ColBERT compares

ColBERT alongside other open-source rerank, search & hybrid tools AI/TLDR tracks, ranked by GitHub stars.