What is LLM Wiki Compiler?
LLM Wiki Compiler — the npm package `llm-wiki-compiler`, whose executable is `llmwiki` — is a knowledge compiler: raw sources go in, an interlinked markdown wiki comes out. It implements Andrej Karpathy's LLM Wiki pattern, moving work from query time to compile time. It sits in the Notes & Knowledge Bases category of the AI Productivity & Writing tools and is open-source, written in TypeScript and licensed MIT.
Overview
LLM Wiki Compiler — the npm package `llm-wiki-compiler`, whose executable is `llmwiki` — is a knowledge compiler: raw sources go in, an interlinked markdown wiki comes out. It implements Andrej Karpathy's LLM Wiki pattern, moving work from query time to compile time. Instead of retrieving raw chunks and asking a model to reconstruct relationships for every question, as traditional RAG does, it first turns papers, notes, READMEs, transcripts, PDFs, images or web pages into typed, interlinked pages that carry citations, metadata and review state.
A two-phase LLM pipeline extracts concepts, then generates pages of four kinds — `concept`, `entity`, `comparison` and `overview` — whose paragraphs cite source files and line ranges that `llmwiki lint` validates. Compilation is incremental, so unchanged sources do not flow back through the model, and stale or orphaned pages can be detected and repaired with `llmwiki refresh --stale`. Queries, evidence packs for agents, exports and MCP tools all operate over the compiled wiki, using semantic chunk search, BM25 reranking and wikilink graph expansion.
Beyond the default concepts-and-queries layout, a validated `.llmwiki/profile.json` (Configurable Lifecycle Profiles) can declare entity types, typed relations, lifecycle states, review gates and workflows that the runtime enforces rather than leaving to prompts. Built-in templates include `autosci` for research and `newsroom` for editorial work. The project names its sweet spot as a durable project or domain wiki — research folders, codebase docs, team handbooks, decision logs — and says it is not meant as a general static-site generator or a replacement for ad-hoc search over fast-changing raw logs. It is written in TypeScript under the MIT license and requires Node.js 24 or newer.
What it does
- Two-phase compile that produces typed `concept`, `entity`, `comparison` and `overview` pages with citations to source files and line ranges
- Hybrid retrieval — semantic chunk search, BM25 reranking and wikilink graph expansion — behind `llmwiki query` and citation-aware `llmwiki context` evidence packs
- Local read-only viewer (`llmwiki view`) with search, graph exploration, source-freshness badges, citation chips and four themes
- Review queues, freshness checks, `llmwiki lint` and an `llmwiki eval` harness that can enforce quality thresholds in CI
- MCP server (`llmwiki serve`), a TypeScript SDK (`createWiki({ root })`) and exports to Open Knowledge Format, JSON, JSON-LD, GraphML, Marp and `llms.txt`
- Provider choice via environment variables: Anthropic by default, plus Claude Agent SDK login, OpenAI Codex CLI login, OpenAI-compatible servers, Ollama, GitHub Copilot, Atlas Cloud and OrcaRouter
Getting started
llmwiki needs Node.js 24 or newer and an LLM provider. Anthropic is the default; set `LLMWIKI_PROVIDER` to use another.
Install the CLI and set a provider
The npm package is `llm-wiki-compiler`; the executable it installs is `llmwiki`.
npm install -g llm-wiki-compiler
export ANTHROPIC_API_KEY=sk-...
# or choose another provider:
# export LLMWIKI_PROVIDER=openai
# export OPENAI_API_KEY=sk-...Compile a first wiki and ask it a question
`quickstart` ingests one source (a URL or a local file), compiles pages and can open the viewer. `view --open` starts the read-only local browser UI.
mkdir my-wiki && cd my-wiki
llmwiki quickstart https://en.wikipedia.org/wiki/Andrej_Karpathy
llmwiki query "What terms did Andrej coin?"
llmwiki view --openGrow and maintain it
Add more sources, recompile incrementally, check citations and freshness, and save useful answers back into the wiki. Inside an existing project, `llmwiki next` suggests the safest next action.
llmwiki ingest <url-or-file>
llmwiki compile
llmwiki lint
llmwiki query "question" --save
llmwiki refresh --stale --dry-runHand it to an agent or your own code
Run the MCP server for MCP-compatible agents, or drive the same operations from TypeScript through the SDK.
import { createWiki } from "llm-wiki-compiler";
const wiki = createWiki({ root: "/path/to/wiki-project" });
await wiki.ingest({ source: "./notes.md" });
await wiki.compile();
const answer = await wiki.query({ question: "What changed?" });Commands and code are distilled from the project's own documentation — always check the official repo for the latest.
When to use it
- Turn a folder of research papers, notes and PDFs into a cited, browsable wiki that grows as sources are added
- Give coding or research agents a stable, citation-aware context pack over MCP instead of a pile of loose files
- Keep a team handbook, design notes or decision log as compiled pages with review gates and stale-page repair
- Run a typed research or editorial workflow from the built-in `autosci` or `newsroom` templates
How LLM Wiki Compiler compares
LLM Wiki Compiler alongside other open-source notes & knowledge bases tools AI/TLDR tracks, ranked by GitHub stars.
| Tool | Stars | What it does |
|---|---|---|
| SiYuan | ★ 47k | A self-hosted, block-level knowledge base with Markdown WYSIWYG editing, SQL queries over blocks, spaced-repetition flashcards and AI writing and Q&A through an OpenAI-compatible API. |
| notebooklm-py | ★ 20.1k | An unofficial Python API, CLI and MCP server for Google's Gemini Notebook (formerly NotebookLM): bulk source import, cited Q&A, and batch export of audio, video, slides, quizzes, mind maps and data tables. |
| MineContext | ★ 5.5k | Local-first desktop app that captures and understands your screen context, then proactively delivers summaries, to-do lists and activity records. |
| Smart Connections | ★ 5.5k | Obsidian plugin that surfaces semantically related notes and excerpts as you write, using a bundled local embedding model that needs no API key and works offline and on mobile. |
| OpenKnowledge | ★ 4.8k | A local-first WYSIWYG markdown IDE and LLM wiki that keeps a knowledge base as plain files in git and exposes it to Claude, Codex and OpenCode over MCP. |
| ReadAny | ★ 2.9k | A local-first e-book reader for desktop and mobile that indexes your library into a local vector store, then answers questions about a book with hybrid vector + BM25 retrieval, alongside highlights, notes, TTS and WebDAV sync. |
| LLM Wiki Compiler | ★ 2.2k | Compile raw sources into an interlinked, citation-traceable markdown wiki |
| LLM Wiki | ★ 1.7k | An open-source take on Karpathy's LLM Wiki that has Claude compile your documents, web clips and highlights into a cited, cross-linked Markdown wiki over MCP, with a web app, graph viewer and Chrome clipper. |