llmspeed

// Overview

llmspeed turns a tokens-per-second number into something you can feel. You pick a GPU, Mac or CPU and a model, and the page streams an answer at the speed that setup is estimated to reach, after the wait for the first token that a prompt of that length would cause. A race mode runs two setups side by side, for example an RTX 5090 against a Mac Studio, and every setup or race has a shareable link.

Each figure carries a trust label. Measured numbers come from benchmarks on that exact hardware, model and quantization, collected from r/LocalLLaMA, localscore.ai, llama.cpp discussions and contributor pull requests. Derived numbers are interpolated from similar measured setups, and Projected numbers are calculated from specs for announced hardware. Where no benchmark exists, generation speed is modelled as memory-bandwidth bound (only active parameters count for Mixture-of-Experts models), prompt processing as compute bound, and partial offload splits work between GPU and system-RAM bandwidth.

The app also checks memory fit: it counts weights, KV cache and runtime overhead against about 75% of unified memory on Macs and 93% of VRAM on discrete GPUs, and says whether each quantization fits, needs partial offload or will not run. The project is a fully static Vite, Preact and TypeScript site under the MIT license. Its README warns that estimates can vary 10–30% with drivers, engine versions, flash attention, power limits and thermals, and that multi-GPU splits are not simulated.

// What it does

• Real-time simulation that streams text at the predicted tokens per second after an estimated time to first token
• Race mode that runs two hardware setups side by side
• Memory-fit check per quantization: fits, partial offload to system RAM, or will not run
• Coverage of 30 GPUs, Macs and CPUs by 15 popular models across CUDA, Metal, ROCm, Vulkan and CPU backends
• Measured, Derived and Projected trust labels on every estimate
• Shareable links to any setup or race
• Community benchmark submissions through pull requests with llama-bench output

// Getting started

llmspeed runs in the browser at llmspeed.dev, so there is nothing to install to use it. The repository is a static Vite site you can run locally, and it accepts new benchmark measurements by pull request.

npm install
npm run dev      # local dev server
npm run build    # production build + prerendered landing pages + sitemap

llama-bench -m your-model.gguf -p 512 -n 128

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Compare how a model would feel on a discrete GPU versus a Mac before buying either
• Check which quantization of a model fits in a machine's memory, and when it would spill to system RAM
• Share a link to a specific hardware-and-model setup when advising someone on a local-LLM build
• Contribute measured llama-bench numbers so other users see real figures instead of estimates

// How llmspeed compares

llmspeed alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.