# Model2Vec

> Distill any sentence transformer into a tiny, fast static embedding model

- Category: Retrieval & Knowledge (RAG) / Embedding Models & Inference
- Language: Python
- Repository: https://github.com/MinishLab/model2vec
- Website: https://minishlab.github.io/
- Canonical page: https://ai-tldr.dev/tools/model2vec/

## In short

Model2Vec is a Python library that turns a regular sentence transformer into a small, static embedding model. Instead of running a neural network for every input, a static model looks up pre-computed token embeddings, which makes it much smaller and faster while keeping most of the quality. It sits in the Embedding Models & Inference category of the Retrieval & Knowledge (RAG) tools and is open-source, written in Python.

## Overview

Model2Vec is a Python library that turns a regular sentence transformer into a small, static embedding model. Instead of running a neural network for every input, a static model looks up pre-computed token embeddings, which makes it much smaller and faster while keeping most of the quality. The maintainers report size reductions of up to 50x and speed-ups of up to 500x, with a small drop in performance.

It is aimed at developers who need text embeddings but want to run them on CPU without the overhead of a full transformer. You can use the ready-made Potion models from the Hugging Face hub, or distill your own static model from a sentence transformer in about 30 seconds on a CPU.

As an embedding model tool, it fits tasks like text classification, retrieval, clustering, and building RAG systems. The embeddings it produces are standard vectors you can feed into any of those pipelines.

## What it does

- Loads pre-trained Potion static models directly from the Hugging Face hub, ready to use
- Distills your own static model from any sentence transformer in about 30 seconds on a CPU
- Runs on CPU without a neural network at inference, so embeddings are small and fast
- Produces both pooled embeddings and per-token embedding sequences via encode and encode_as_sequence
- Optional training extra lets you fine-tune classification models on top of a distilled or pre-trained model
- Supports BPE and Unigram tokenizer backends, plus quantization and dimensionality reduction to shrink models further

## Getting started

Install the base package, then load a Potion model and create embeddings. If you want to build your own static model, install the distillation extra.

1. Install the base package

   Install the lightweight base package with pip.

```bash
pip install model2vec
```

2. Load a model and make embeddings

   Load a pre-trained Potion model from the Hugging Face hub and encode some text.

```python
from model2vec import StaticModel

# Load a model from the HuggingFace hub (in this case the potion-base-32M model)
model = StaticModel.from_pretrained("minishlab/potion-base-32M")

# Make embeddings
embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])

# Make sequences of token embeddings
token_embeddings = model.encode_as_sequence(["It's dangerous to go alone!", "It's a secret to everybody."])
```

3. Distill your own model (optional)

   Install the distillation extra, then distill a static model from a sentence transformer in about 30 seconds on a CPU.

```bash
pip install model2vec[distill]
```

4. Run the distillation

   Distill a sentence transformer and save the resulting static model.

```python
from model2vec.distill import distill

# Distill a Sentence Transformer model, in this case the BAAI/bge-base-en-v1.5 model
m2v_model = distill(model_name="BAAI/bge-base-en-v1.5")

# Save the model
m2v_model.save_pretrained("m2v_model")
```

## When to use it

- Add fast, CPU-only text embeddings to a retrieval or RAG pipeline without hosting a full transformer
- Shrink an existing sentence transformer into a smaller static model for resource-constrained or edge deployments
- Generate vectors for text classification and clustering at high throughput
- Embed multilingual text using the pre-trained potion-multilingual-128M model