llmfit

// Overview

llmfit answers the question every local-model user hits first: which of these will actually run here? It inspects your CPU cores, system RAM, discrete and integrated GPUs, VRAM and accelerator configuration — covering NVIDIA CUDA, Apple Silicon, AMD ROCm and Intel OneAPI, including unified-memory architectures — and then scores candidate models against what it found.

The scoring is not a single pass/fail. A compatibility engine reads each model's parameter count, context length and quantization format (GGUF, AWQ, GPTQ, EXL2) to project memory footprint and tokens-per-second, then rates every model on quality, speed, fit and context so you can see the trade-off rather than just a yes or no. It handles multi-GPU setups, MoE architectures and dynamic quantization selection, and knows the local runtime providers you would actually serve through: Ollama, llama.cpp, MLX, Docker Model Runner and LM Studio.

Estimates can be replaced with measurements. From the TUI you can download a model, serve it, benchmark real tokens per second on your own machine, and submit the result back to the project as a pull request — no GitHub CLI or third-party account involved. Runs are saved locally first and your own numbers override the estimates in your fit table; merged submissions ship in the next release, so anyone on identical hardware sees measured figures before they benchmark anything themselves. Beyond the interactive TUI there is a classic CLI mode, a web dashboard, and a REST API (/api/v1/system, /api/v1/models) for wiring the same data into orchestrators and deployment pipelines.

// What it does

• Hardware auto-detection for CPU cores, RAM, discrete/integrated GPUs, VRAM and unified memory across CUDA, Apple Silicon, ROCm and OneAPI
• Compatibility engine projecting memory footprint and tokens-per-second from parameter count, context length and quantization (GGUF, AWQ, GPTQ, EXL2)
• Scores every model on quality, speed, fit and context, with multi-GPU, MoE and dynamic quantization support
• Built-in benchmarking: download, serve and measure real tok/s, then contribute results upstream as a PR from the TUI
• Interactive TUI plus a classic CLI mode and a web dashboard
• REST endpoints (/api/v1/system, /api/v1/models) for integration into orchestrators and automated deployments
• Knows local runtime providers — Ollama, llama.cpp, MLX, Docker Model Runner and LM Studio

// Getting started

llmfit is a single Rust binary with packages on most platforms. It runs macOS (Apple Silicon and Intel), Linux (x86_64 and ARM64) and Windows (x86_64); release binaries are code-signed.

brew install AlexsJones/llmfit/llmfit

# or, without sudo:
curl -fsSL https://llmfit.axjns.dev/install.sh | sh -s -- --local

scoop install llmfit

uv tool install -U llmfit

# or run it once:
uvx llmfit

docker run -it --rm ghcr.io/alexsjones/llmfit --tui

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Decide which open-weight model and quantization to download before spending bandwidth on a file that will not fit
• Size a workstation or GPU purchase against the models you actually intend to run
• Replace estimated throughput with a measured benchmark on your own hardware, and share it back to the project
• Expose hardware and model-fit data to an orchestrator or deployment pipeline through the REST API

// How llmfit compares

llmfit alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.