# LLM Wiki Compiler

> Compile raw sources into an interlinked, citation-traceable markdown wiki

- Category: AI Productivity & Writing / Notes & Knowledge Bases
- Language: TypeScript
- Licence: MIT
- Repository: https://github.com/atomicstrata/llm-wiki-compiler
- Website: https://llmwiki.atomicstrata.ai
- Canonical page: https://ai-tldr.dev/tools/llm-wiki-compiler/

## In short

LLM Wiki Compiler — the npm package `llm-wiki-compiler`, whose executable is `llmwiki` — is a knowledge compiler: raw sources go in, an interlinked markdown wiki comes out. It implements Andrej Karpathy's LLM Wiki pattern, moving work from query time to compile time. It sits in the Notes & Knowledge Bases category of the AI Productivity & Writing tools and is open-source, written in TypeScript and licensed MIT.

## Overview

LLM Wiki Compiler — the npm package `llm-wiki-compiler`, whose executable is `llmwiki` — is a knowledge compiler: raw sources go in, an interlinked markdown wiki comes out. It implements Andrej Karpathy's LLM Wiki pattern, moving work from query time to compile time. Instead of retrieving raw chunks and asking a model to reconstruct relationships for every question, as traditional RAG does, it first turns papers, notes, READMEs, transcripts, PDFs, images or web pages into typed, interlinked pages that carry citations, metadata and review state.

A two-phase LLM pipeline extracts concepts, then generates pages of four kinds — `concept`, `entity`, `comparison` and `overview` — whose paragraphs cite source files and line ranges that `llmwiki lint` validates. Compilation is incremental, so unchanged sources do not flow back through the model, and stale or orphaned pages can be detected and repaired with `llmwiki refresh --stale`. Queries, evidence packs for agents, exports and MCP tools all operate over the compiled wiki, using semantic chunk search, BM25 reranking and wikilink graph expansion.

Beyond the default concepts-and-queries layout, a validated `.llmwiki/profile.json` (Configurable Lifecycle Profiles) can declare entity types, typed relations, lifecycle states, review gates and workflows that the runtime enforces rather than leaving to prompts. Built-in templates include `autosci` for research and `newsroom` for editorial work. The project names its sweet spot as a durable project or domain wiki — research folders, codebase docs, team handbooks, decision logs — and says it is not meant as a general static-site generator or a replacement for ad-hoc search over fast-changing raw logs. It is written in TypeScript under the MIT license and requires Node.js 24 or newer.

## What it does

- Two-phase compile that produces typed `concept`, `entity`, `comparison` and `overview` pages with citations to source files and line ranges
- Hybrid retrieval — semantic chunk search, BM25 reranking and wikilink graph expansion — behind `llmwiki query` and citation-aware `llmwiki context` evidence packs
- Local read-only viewer (`llmwiki view`) with search, graph exploration, source-freshness badges, citation chips and four themes
- Review queues, freshness checks, `llmwiki lint` and an `llmwiki eval` harness that can enforce quality thresholds in CI
- MCP server (`llmwiki serve`), a TypeScript SDK (`createWiki({ root })`) and exports to Open Knowledge Format, JSON, JSON-LD, GraphML, Marp and `llms.txt`
- Provider choice via environment variables: Anthropic by default, plus Claude Agent SDK login, OpenAI Codex CLI login, OpenAI-compatible servers, Ollama, GitHub Copilot, Atlas Cloud and OrcaRouter

## Getting started

llmwiki needs Node.js 24 or newer and an LLM provider. Anthropic is the default; set `LLMWIKI_PROVIDER` to use another.

1. Install the CLI and set a provider

   The npm package is `llm-wiki-compiler`; the executable it installs is `llmwiki`.

```bash
npm install -g llm-wiki-compiler

export ANTHROPIC_API_KEY=sk-...
# or choose another provider:
# export LLMWIKI_PROVIDER=openai
# export OPENAI_API_KEY=sk-...
```

2. Compile a first wiki and ask it a question

   `quickstart` ingests one source (a URL or a local file), compiles pages and can open the viewer. `view --open` starts the read-only local browser UI.

```bash
mkdir my-wiki && cd my-wiki
llmwiki quickstart https://en.wikipedia.org/wiki/Andrej_Karpathy
llmwiki query "What terms did Andrej coin?"
llmwiki view --open
```

3. Grow and maintain it

   Add more sources, recompile incrementally, check citations and freshness, and save useful answers back into the wiki. Inside an existing project, `llmwiki next` suggests the safest next action.

```bash
llmwiki ingest <url-or-file>
llmwiki compile
llmwiki lint
llmwiki query "question" --save
llmwiki refresh --stale --dry-run
```

4. Hand it to an agent or your own code

   Run the MCP server for MCP-compatible agents, or drive the same operations from TypeScript through the SDK.

```ts
import { createWiki } from "llm-wiki-compiler";

const wiki = createWiki({ root: "/path/to/wiki-project" });
await wiki.ingest({ source: "./notes.md" });
await wiki.compile();
const answer = await wiki.query({ question: "What changed?" });
```

## When to use it

- Turn a folder of research papers, notes and PDFs into a cited, browsable wiki that grows as sources are added
- Give coding or research agents a stable, citation-aware context pack over MCP instead of a pile of loose files
- Keep a team handbook, design notes or decision log as compiled pages with review gates and stale-page repair
- Run a typed research or editorial workflow from the built-in `autosci` or `newsroom` templates