autoresearch

// Overview

autoresearch, published by Andrej Karpathy in March 2026, hands a coding agent a small but real LLM training setup and lets it run experiments without you. The agent edits the training code, trains for five minutes, checks whether the result improved, keeps or discards the change, and repeats. You come back to a log of experiments and, hopefully, a better model. The training code is a simplified single-GPU implementation of nanochat.

The interesting inversion is what you edit. As a researcher you do not touch the Python files; you write `program.md`, the Markdown file that gives the agent its context and sets up what Karpathy calls your autonomous research org. The default `program.md` is deliberately a bare-bones baseline — the point is that you iterate on it over time to find the research-org instructions that make progress fastest, add more agents to the mix, and so on. Karpathy describes it as essentially a very lightweight skill.

Only three files matter. `prepare.py` holds fixed constants, one-time data preparation and runtime utilities and is not modified. `train.py` — the full GPT model, a Muon plus AdamW optimizer, and the training loop — is the single file the agent edits, and everything in it is fair game: architecture, hyperparameters, optimizer, batch size. Training always runs for a fixed five minutes of wall clock excluding startup and compilation, which makes runs on the same machine directly comparable no matter what the agent changed, and the metric is `val_bpb`, validation bits per byte, which is vocabulary-size independent so architectural changes compete fairly. The trade is that your results are not comparable to someone else's on different hardware.

// What it does

• A single file for the agent to edit — model, optimizer and training loop all live in train.py, with prepare.py held fixed
• program.md as the human-editable control surface: the agent's instructions, not the Python, are what you iterate on
• Fixed five-minute wall-clock training budget, so roughly 12 experiments an hour and about 100 overnight, all directly comparable
• val_bpb (validation bits per byte) as the metric — vocabulary-size independent, so architecture changes are judged fairly
• Self-contained: PyTorch and a few small packages, no distributed training and no complex configs
• Simplified single-GPU nanochat as the training code, small enough that diffs stay reviewable
• Documented knobs for scaling down to smaller machines — dataset choice, vocab_size, MAX_SEQ_LEN, DEPTH, WINDOW_PATTERN and batch size

// Getting started

autoresearch needs a single NVIDIA GPU (tested on an H100), Python 3.10 or newer, and the uv project manager. Community forks cover macOS, Windows and AMD.

curl -LsSf https://astral.sh/uv/install.sh | sh

uv sync
uv run prepare.py     # one-time, about 2 minutes
uv run train.py       # one experiment, about 5 minutes

Hi have a look at program.md and let's kick off a new experiment! let's do the setup first.

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Reach for it to run an overnight sweep of architecture and optimizer ideas without babysitting each one
• Reach for it as a testbed for prompt and skill design, where program.md is the thing under study rather than the model
• Reach for it to teach the training loop, since the whole model, optimizer and loop fit in one readable file
• Reach for it when you want experiments that are fairly comparable to each other on one machine, by construction

// How autoresearch compares

autoresearch alongside other open-source autonomous coding agents tools AI/TLDR tracks, ranked by GitHub stars.