█

AI/TLDR

RankLLM

Rerank retrieved passages with listwise LLM rankers and open-source models

GitHub★ 656Websiterankllm.ai
Rerank, Search & HybridOpen source
Language
Python
$uv venv --python 3.11

Overview

RankLLM is a Python toolkit from the Castorini group for reranking retrieved passages in a search or RAG pipeline. It offers a suite of rerankers: pointwise models like MonoT5, pairwise models like DuoT5, and listwise rankers built around open-source LLMs. It also supports proprietary listwise variants such as RankGPT and RankGemini.

It is aimed at developers and researchers who already have a first-stage retriever (for example BM25) and want to reorder the top candidates for better relevance. You can run it as a command-line tool, as an HTTP or MCP server, or call it directly from Python.

Within the reranker and hybrid-search category, RankLLM focuses on the second stage of retrieval. It integrates with vLLM and SGLang for serving open-source models, and can also reorder candidates using first-token logits to keep inference cheaper.

What it does

  • Pointwise (MonoT5), pairwise (DuoT5), and listwise rerankers in one toolkit
  • Works with open-source LLMs through vLLM and SGLang backends
  • Supports proprietary listwise rerankers including RankGPT and RankGemini
  • First-token-logit reranking option to improve inference efficiency
  • rank-llm CLI for rerank, evaluate, prompt listing, and HTTP/MCP serving
  • Custom prompt templates via YAML and an OpenRouter API integration

Getting started

Install the published package into an isolated environment, then rerank a dataset with the rank-llm CLI or from Python.

Install from PyPI

Create a virtual environment with uv and install the rank-llm package. A conda/pip fallback is also supported.

bashbash
uv venv --python 3.11
source .venv/bin/activate
uv pip install rank-llm

Add the stack you need

Optional extras pull in only the dependencies for your workflow, such as local Hugging Face models, vLLM, or hosted providers.

bashbash
pip install -e ".[local]"   # or [vllm], [openai], [pyserini], [all]

Rerank from the CLI

The rank-llm command reranks a dataset's top candidates. This example retrieves with BM25 and reranks with RankZephyr.

bashbash
rank-llm rerank --model-path castorini/rank_zephyr_7b_v1_full --dataset dl20 \
  --retrieval-method bm25 --top-k-candidates 100

Use it from Python

Import a reranker directly to plug RankLLM into your own retrieval pipeline. Sample scripts live under src/rank_llm/demo.

pythonpython
from rank_llm.rerank import Reranker
from rank_llm.rerank.listwise import ZephyrReranker
from rank_llm.retrieve.retriever import RetrievalMethod, Retriever

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

When to use it

  • Improve relevance in a RAG pipeline by reordering the top passages before sending them to a generator
  • Reorder BM25 or dense-retrieval candidates with a listwise LLM reranker like RankZephyr
  • Benchmark and compare pointwise, pairwise, and listwise rerankers on TREC DL datasets
  • Expose reranking to other tools through an HTTP or MCP server

How RankLLM compares

RankLLM alongside other open-source rerank, search & hybrid tools AI/TLDR tracks, ranked by GitHub stars.

ToolStarsWhat it does
Elasticsearch★ 77.9kDistributed search and analytics engine with a built-in vector database for dense/sparse embeddings and hybrid keyword-plus-semantic retrieval.
Meilisearch Cloud★ 59.3kManaged cloud for the Meilisearch engine, combining fast full-text search with hybrid, semantic, and multimodal vector search.
Typesense Cloud★ 26.6kManaged hosting for the Typesense search engine, offering typo-tolerant keyword search plus vector and semantic search via a simple API.
Tantivy★ 16.1kA fast full-text search engine library in Rust that provides BM25 keyword search for the lexical half of hybrid retrieval.
FlagEmbedding★ 12.2kBAAI's retrieval toolkit that provides the BGE embedding and cross-encoder reranker models used widely in RAG pipelines.
Vespa★ 7.1kA search and serving engine that natively combines vector, keyword (BM25), and structured search with built-in ranking for large-scale retrieval.
RAGatouille★ 4kA wrapper that makes it easy to train and use ColBERT late-interaction retrieval inside RAG pipelines.
RankLLM★ 656Rerank retrieved passages with listwise LLM rankers and open-source models