# RankLLM

> Rerank retrieved passages with listwise LLM rankers and open-source models

- Category: Retrieval & Knowledge (RAG) / Rerank, Search & Hybrid
- Language: Python
- Repository: https://github.com/castorini/rank_llm
- Website: https://rankllm.ai
- Canonical page: https://ai-tldr.dev/tools/rankllm/

## In short

RankLLM is a Python toolkit from the Castorini group for reranking retrieved passages in a search or RAG pipeline. It offers a suite of rerankers: pointwise models like MonoT5, pairwise models like DuoT5, and listwise rankers built around open-source LLMs. It sits in the Rerank, Search & Hybrid category of the Retrieval & Knowledge (RAG) tools and is open-source, written in Python.

## Overview

RankLLM is a Python toolkit from the Castorini group for reranking retrieved passages in a search or RAG pipeline. It offers a suite of rerankers: pointwise models like MonoT5, pairwise models like DuoT5, and listwise rankers built around open-source LLMs. It also supports proprietary listwise variants such as RankGPT and RankGemini.

It is aimed at developers and researchers who already have a first-stage retriever (for example BM25) and want to reorder the top candidates for better relevance. You can run it as a command-line tool, as an HTTP or MCP server, or call it directly from Python.

Within the reranker and hybrid-search category, RankLLM focuses on the second stage of retrieval. It integrates with vLLM and SGLang for serving open-source models, and can also reorder candidates using first-token logits to keep inference cheaper.

## What it does

- Pointwise (MonoT5), pairwise (DuoT5), and listwise rerankers in one toolkit
- Works with open-source LLMs through vLLM and SGLang backends
- Supports proprietary listwise rerankers including RankGPT and RankGemini
- First-token-logit reranking option to improve inference efficiency
- rank-llm CLI for rerank, evaluate, prompt listing, and HTTP/MCP serving
- Custom prompt templates via YAML and an OpenRouter API integration

## Getting started

Install the published package into an isolated environment, then rerank a dataset with the rank-llm CLI or from Python.

1. Install from PyPI

   Create a virtual environment with uv and install the rank-llm package. A conda/pip fallback is also supported.

```bash
uv venv --python 3.11
source .venv/bin/activate
uv pip install rank-llm
```

2. Add the stack you need

   Optional extras pull in only the dependencies for your workflow, such as local Hugging Face models, vLLM, or hosted providers.

```bash
pip install -e ".[local]"   # or [vllm], [openai], [pyserini], [all]
```

3. Rerank from the CLI

   The rank-llm command reranks a dataset's top candidates. This example retrieves with BM25 and reranks with RankZephyr.

```bash
rank-llm rerank --model-path castorini/rank_zephyr_7b_v1_full --dataset dl20 \
  --retrieval-method bm25 --top-k-candidates 100
```

4. Use it from Python

   Import a reranker directly to plug RankLLM into your own retrieval pipeline. Sample scripts live under src/rank_llm/demo.

```python
from rank_llm.rerank import Reranker
from rank_llm.rerank.listwise import ZephyrReranker
from rank_llm.retrieve.retriever import RetrievalMethod, Retriever
```

## When to use it

- Improve relevance in a RAG pipeline by reordering the top passages before sending them to a generator
- Reorder BM25 or dense-retrieval candidates with a listwise LLM reranker like RankZephyr
- Benchmark and compare pointwise, pairwise, and listwise rerankers on TREC DL datasets
- Expose reranking to other tools through an HTTP or MCP server