Langchain-Chatchat

// Overview

Langchain-Chatchat, formerly Langchain-ChatGLM, is an open-source question-answering application built on the LangChain concept. Its goal is a friendly, offline-operable knowledge base Q&A solution that works well for Chinese-language scenarios and runs fully on open-source models — so the whole pipeline, including the model, can be deployed privately with no data leaving your machines.

The retrieval pipeline is the classic RAG loop: load files, read text, split it into chunks, vectorise the chunks, vectorise the question, match the top-k most similar chunks, then append the matched text as context to the prompt and hand it to the LLM. Version 0.3.x builds its core features on an Agent, and adds database conversations, arXiv document conversations, Wolfram queries and text-to-image on top of the LLM, knowledge base, search-engine and file conversation modes. File RAG supports BM25 and KNN retrieval, not only vector search.

From 0.3.0 the project stopped loading models from a local path itself and instead integrates with mainstream inference frameworks — Xinference, Ollama, LocalAI, FastChat and One API — all through an OpenAI-compatible interface. That means it runs across CPU, GPU, NPU and MPS hardware, and can equally point at a hosted API such as OpenAI, Azure OpenAI, Anthropic or Zhipu. It is a Python package under Apache-2.0 with a Streamlit WebUI and a FastAPI backend.

// What it does

• Offline, fully open-source RAG stack — LLM, embeddings and vector store can all run privately with no external API calls
• Multiple chat modes: plain LLM, local knowledge base, search engine, single-file, database and arXiv document conversations
• File RAG with BM25 + KNN retrieval alongside vector similarity search
• Agent-driven tool calling, with manual tool selection for models that lack reliable agent behaviour
• Pluggable inference backends — Xinference, Ollama, LocalAI, FastChat and One API — all via the OpenAI SDK interface
• Local knowledge base management and a multi-session WebUI with custom system prompts

// Getting started

From version 0.3.0 Langchain-Chatchat installs as a Python library and is configured from the command line. It supports Python 3.8-3.11 on Windows, macOS and Linux. Start an inference framework first — the steps below follow the project's own quick start.

pip install langchain-chatchat -U
# or, when pairing it with Xinference:
pip install "langchain-chatchat[xinference]" -U

chatchat-config model --set_model_platforms "[{
    \"platform_name\": \"xinference\",
    \"platform_type\": \"xinference\",
    \"api_base_url\": \"http://127.0.0.1:9997/v1\",
    \"api_key\": \"EMPT\",
    \"api_concurrencies\": 5,
    \"llm_models\": [\"autodl-tmp-glm-4-9b-chat\"],
    \"embed_models\": [\"bge-large-zh-v1.5\"],
    \"image_models\": [],
    \"reranking_models\": [],
    \"speech2text_models\": [],
    \"tts_models\": []
}]"

chatchat-kb -r
# or with an explicit embedding model:
chatchat-kb -r --embed-model=text-embedding-3-small

chatchat -a

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Stand up a private document Q&A service in an air-gapped or compliance-constrained environment where no data may leave the network
• Build a Chinese-language knowledge base assistant on open-source models such as GLM-4-Chat or Qwen2-Instruct
• Prototype a RAG pipeline with a WebUI before committing to a bespoke stack, swapping the inference backend as needs change
• Give non-developers a managed local knowledge base with upload, re-indexing and multi-session chat out of the box

// How Langchain-Chatchat compares

Langchain-Chatchat alongside other open-source rag frameworks & platforms tools AI/TLDR tracks, ranked by GitHub stars.