turbovec

// Overview

turbovec is an embedded vector index written in Rust and shipped with Python bindings. It is built on TurboQuant, a quantizer from Google Research that is data-oblivious: it needs no separate training phase and no parameter tuning, so vectors are indexed the moment you add them rather than after a build or retrain step. That makes it suited to corpora that grow continuously, where a train-then-index workflow means periodic rebuilds.

The project's headline claim is memory: a 10-million-document corpus that needs 31 GB of RAM as float32 fits in 4 GB once quantized to 4 bits. Search runs through hand-written SIMD kernels — NEON SDOT/SMMLA on ARM, AVX-512 VNNI and vpermb on x86, with AVX2 and scalar fallbacks. The maintainers benchmark against FAISS IndexPQFastScan at 100K vectors and report wins in every measured configuration, averaging roughly 3.4x at 4-bit and about 20-26% at 2-bit on both architectures.

It runs entirely in-process — there is no managed service and no data leaving your machine or VPC — so it fits air-gapped or privacy-constrained RAG stacks paired with any open-source embedding model. Two index types are exposed: TurboQuantIndex for positional results and IdMapIndex for stable uint64 external ids that survive deletes. Drop-in vector-store adapters are published for LangChain, LlamaIndex, Haystack, and Agno.

// What it does

• Online ingest: vectors are indexed as they are added, with no train step, no parameter tuning, and no rebuilds as the corpus grows
• 2-bit and 4-bit TurboQuant compression, cutting a 31 GB float32 corpus of 10M documents to about 4 GB
• Hand-written SIMD search kernels for ARM (NEON SDOT/SMMLA) and x86 (AVX-512 VNNI, vpermb), with AVX2 and scalar fallbacks
• Incremental crash-safe persistence: sync(path) writes only what changed since the last sync, one fsync per call
• Filtered search: pass an id allowlist to search() and the kernel honours it inside the SIMD scan, so selective filters skip work instead of over-fetching
• IdMapIndex for stable uint64 external ids with O(1) remove-by-id
• Drop-in replacements for the in-tree vector stores in LangChain, LlamaIndex, Haystack, and Agno

// Getting started

turbovec ships on PyPI for Python and crates.io for Rust. Vectors and queries are 2-D float32 arrays of shape (n, dim); other dtypes are rejected rather than silently converted, so cast with np.asarray(x, dtype=np.float32) first if needed.

pip install turbovec

from turbovec import TurboQuantIndex

index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors)
index.add(more_vectors)

scores, indices = index.search(query, k=10)

index.write("my_index.tv")
loaded = TurboQuantIndex.load("my_index.tv")

index.sync("my_index.tv")   # after more changes: durable incremental save

import numpy as np
from turbovec import IdMapIndex

index = IdMapIndex(dim=1536, bit_width=4)
index.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64))

scores, ids = index.search(query, k=10)   # ids are your uint64 external ids
index.remove(1002)                         # O(1) by id

index.write("my_index.tvim")

import numpy as np
from turbovec import IdMapIndex

idx = IdMapIndex(dim=1536, bit_width=4)
idx.add_with_ids(vectors, ids)

# Stage 1: external system narrows to candidate ids.
allowed = np.array(db.execute("SELECT id FROM docs WHERE tenant=?", (t,)).fetchall(),
                   dtype=np.uint64)

# Stage 2: dense rerank within the candidate set.
scores, ids = idx.search(query, k=10, allowlist=allowed)

cargo add turbovec

pip install turbovec[langchain]     # replaces InMemoryVectorStore
pip install turbovec[llama-index]   # replaces SimpleVectorStore
pip install turbovec[haystack]      # replaces InMemoryDocumentStore
pip install turbovec[agno]          # replaces agno.vectordb.lancedb.LanceDb

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Run RAG retrieval fully locally or air-gapped, where sending embeddings to a managed vector service is not an option
• Fit a large embedding corpus in memory on one machine instead of provisioning a cluster, using 2-bit or 4-bit quantization
• Index a continuously growing corpus without periodic retrain-and-rebuild cycles
• Combine an existing SQL, BM25, or permissions filter with dense reranking by passing an id allowlist into search
• Replace the in-memory reference vector store in a LangChain, LlamaIndex, Haystack, or Agno pipeline without rewriting it

// How turbovec compares

turbovec alongside other open-source vector databases tools AI/TLDR tracks, ranked by GitHub stars.