# turbovec

> A Rust vector index built on TurboQuant, with Python bindings and no training step

- Category: Retrieval & Knowledge (RAG) / Vector Databases
- Language: Rust
- Licence: MIT
- Repository: https://github.com/RyanCodrai/turbovec
- Canonical page: https://ai-tldr.dev/tools/turbovec/

## In short

turbovec is an embedded vector index written in Rust and shipped with Python bindings. It sits in the Vector Databases category of the Retrieval & Knowledge (RAG) tools and is open-source, written in Rust and licensed MIT. To get started: install the Python package.

## Overview

turbovec is an embedded vector index written in Rust and shipped with Python bindings. It is built on TurboQuant, a quantizer from Google Research that is data-oblivious: it needs no separate training phase and no parameter tuning, so vectors are indexed the moment you add them rather than after a build or retrain step. That makes it suited to corpora that grow continuously, where a train-then-index workflow means periodic rebuilds.

The project's headline claim is memory: a 10-million-document corpus that needs 31 GB of RAM as float32 fits in 4 GB once quantized to 4 bits. Search runs through hand-written SIMD kernels — NEON SDOT/SMMLA on ARM, AVX-512 VNNI and vpermb on x86, with AVX2 and scalar fallbacks. The maintainers benchmark against FAISS IndexPQFastScan at 100K vectors and report wins in every measured configuration, averaging roughly 3.4x at 4-bit and about 20-26% at 2-bit on both architectures.

It runs entirely in-process — there is no managed service and no data leaving your machine or VPC — so it fits air-gapped or privacy-constrained RAG stacks paired with any open-source embedding model. Two index types are exposed: TurboQuantIndex for positional results and IdMapIndex for stable uint64 external ids that survive deletes. Drop-in vector-store adapters are published for LangChain, LlamaIndex, Haystack, and Agno.

## What it does

- Online ingest: vectors are indexed as they are added, with no train step, no parameter tuning, and no rebuilds as the corpus grows
- 2-bit and 4-bit TurboQuant compression, cutting a 31 GB float32 corpus of 10M documents to about 4 GB
- Hand-written SIMD search kernels for ARM (NEON SDOT/SMMLA) and x86 (AVX-512 VNNI, vpermb), with AVX2 and scalar fallbacks
- Incremental crash-safe persistence: sync(path) writes only what changed since the last sync, one fsync per call
- Filtered search: pass an id allowlist to search() and the kernel honours it inside the SIMD scan, so selective filters skip work instead of over-fetching
- IdMapIndex for stable uint64 external ids with O(1) remove-by-id
- Drop-in replacements for the in-tree vector stores in LangChain, LlamaIndex, Haystack, and Agno

## Getting started

turbovec ships on PyPI for Python and crates.io for Rust. Vectors and queries are 2-D float32 arrays of shape (n, dim); other dtypes are rejected rather than silently converted, so cast with np.asarray(x, dtype=np.float32) first if needed.

1. Install the Python package

   Install from PyPI. The wheel bundles the compiled Rust extension.

```bash
pip install turbovec
```

2. Build an index and search it

   Create an index for your embedding dimension and bit width, add vectors as they arrive, then search. write() takes a whole-file snapshot; sync() does a durable incremental save afterwards.

```python
from turbovec import TurboQuantIndex

index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors)
index.add(more_vectors)

scores, indices = index.search(query, k=10)

index.write("my_index.tv")
loaded = TurboQuantIndex.load("my_index.tv")

index.sync("my_index.tv")   # after more changes: durable incremental save
```

3. Use stable ids that survive deletes

   IdMapIndex maps your own uint64 ids onto index slots, so search returns your ids and remove() is O(1) by id.

```python
import numpy as np
from turbovec import IdMapIndex

index = IdMapIndex(dim=1536, bit_width=4)
index.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64))

scores, ids = index.search(query, k=10)   # ids are your uint64 external ids
index.remove(1002)                         # O(1) by id

index.write("my_index.tvim")
```

4. Filter to a candidate set (hybrid retrieval)

   Let another system — SQL, BM25, an ACL check, a time window — narrow the candidates, then rerank densely inside that set. Blocks with no allowed slots are short-circuited before any scoring work.

```python
import numpy as np
from turbovec import IdMapIndex

idx = IdMapIndex(dim=1536, bit_width=4)
idx.add_with_ids(vectors, ids)

# Stage 1: external system narrows to candidate ids.
allowed = np.array(db.execute("SELECT id FROM docs WHERE tenant=?", (t,)).fetchall(),
                   dtype=np.uint64)

# Stage 2: dense rerank within the candidate set.
scores, ids = idx.search(query, k=10, allowlist=allowed)
```

5. Or use it from Rust

   The same index types are available as a crate.

```bash
cargo add turbovec
```

6. Swap it into an existing framework

   Each integration keeps the same public surface, persistence semantics, and retriever wiring as the framework's in-tree store, so you change the import and keep the pipeline.

```bash
pip install turbovec[langchain]     # replaces InMemoryVectorStore
pip install turbovec[llama-index]   # replaces SimpleVectorStore
pip install turbovec[haystack]      # replaces InMemoryDocumentStore
pip install turbovec[agno]          # replaces agno.vectordb.lancedb.LanceDb
```

## When to use it

- Run RAG retrieval fully locally or air-gapped, where sending embeddings to a managed vector service is not an option
- Fit a large embedding corpus in memory on one machine instead of provisioning a cluster, using 2-bit or 4-bit quantization
- Index a continuously growing corpus without periodic retrain-and-rebuild cycles
- Combine an existing SQL, BM25, or permissions filter with dense reranking by passing an id allowlist into search
- Replace the in-memory reference vector store in a LangChain, LlamaIndex, Haystack, or Agno pipeline without rewriting it