Hindsight

// Overview

Hindsight is an agent memory system from Vectorize. Where most memory layers concentrate on replaying conversation history, Hindsight is built around the idea that an agent should form durable understanding: memories are pushed through an extraction pipeline and consolidated into evidence-backed beliefs the agent can act on later.

It organizes what it stores into four kinds of memory. World facts describe the world ("the stove gets hot"), experiences record what the agent itself did, observations are consolidated beliefs formed from many memories, and mental models are the synthesized understanding built on top. Memories live in named banks, and recall runs four retrieval strategies in parallel — semantic vector similarity, BM25 keyword matching, entity and temporal graph links, and time-range filtering.

You run it as a server (Docker, pip, Helm, or the managed Hindsight Cloud) and talk to it from Python, Node.js, Go, a CLI, or the REST API. Every server also exposes a Model Context Protocol endpoint per bank, so any MCP client can call retain, recall, and reflect as tools. The code is MIT-licensed; the project reports state-of-the-art results on the LongMemEval benchmark and publishes live benchmark numbers.

// What it does

• Four memory types — world facts, experiences, consolidated observations, and synthesized mental models — instead of a flat conversation log
• Three operations: retain to store, recall to retrieve, and reflect to answer with the memory applied
• Parallel retrieval across semantic, keyword (BM25), graph, and temporal strategies
• An LLM wrapper that adds memory to an existing OpenAI or Anthropic client in about two lines, plus 60+ framework and app integrations
• A built-in MCP endpoint per memory bank, so MCP clients get retain/recall/reflect as tools
• Works with 25+ LLM providers, including fully local ones (Ollama, LM Studio, llama.cpp) and any OpenAI-compatible endpoint
• Deploy with Docker, pip, Helm, an embedded Python server with no separate process, or the managed cloud

// Getting started

Start a server, then connect a client in your language. Docker is the recommended path; the embedded Python option skips the server entirely for local experiments.

export OPENAI_API_KEY=sk-xxx

docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \
  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
  -v hindsight-data:/home/hindsight/.pg0 \
  ghcr.io/vectorize-io/hindsight:latest

pip install hindsight-api
export HINDSIGHT_API_LLM_API_KEY=sk-xxx

hindsight-api

pip install hindsight-client -U                                  # Python
npm install @vectorize-io/hindsight-client                        # Node.js / TypeScript
go get github.com/vectorize-io/hindsight/hindsight-clients/go     # Go

from hindsight_client import Hindsight

client = Hindsight(base_url="http://localhost:8888")

# Retain: store information
client.retain(bank_id="my-bank", content="Alice works at Google as a software engineer")

# Recall: search memories
client.recall(bank_id="my-bank", query="What does Alice do?")

# Reflect: generate a disposition-aware response
client.reflect(bank_id="my-bank", query="Tell me about Alice")

pip install hindsight-litellm

from openai import OpenAI
from hindsight_litellm import wrap_openai

client = wrap_openai(
    OpenAI(),
    bank_id="user-123",
    hindsight_api_url="http://localhost:8888",
)

response = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[{"role": "user", "content": "What do you know about me?"}],
)

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Give an assistant memory that survives sessions, so it recalls a user's facts, preferences, and history without re-prompting
• Let a long-running agent accumulate its own experiences and act on what previously worked or failed
• Give a CLI coding agent per-repository memory built from git history and past sessions
• Expose a shared memory bank to several tools at once through the MCP endpoint

// How Hindsight compares

Hindsight alongside other open-source agent memory tools AI/TLDR tracks, ranked by GitHub stars.