Categories 12
Models & Inference
Local RuntimesModels & Inference · 14 tools
Ollama175kA developer-friendly tool that downloads and runs local LLMs from the terminal with a built-in OpenAI-compatible API.ollama/ollama ↗details →
llama.cpp117kA C/C++ inference engine that runs LLMs in the GGUF format on CPUs, Apple Silicon, and GPUs with low memory use.ggml-org/llama.cpp ↗details →
GPT4All77.4kGPT4All is a free desktop app and Python client that runs large language models locally on your own computer, with no API calls or GPU required.nomic-ai/gpt4all ↗details →
LocalAI47kA self-hosted server that exposes an OpenAI-compatible API for running text, vision, voice, and image models on local hardware.mudler/LocalAI ↗details →
Jan43.1kAn open-source desktop app that runs LLMs fully offline as a ChatGPT-style assistant on your own computer.janhq/jan ↗details →
llamafile25kA Mozilla project that packages a model and its runtime into one executable file you can copy and run on any OS.mozilla-ai/llamafile ↗details →
MLC LLM22.8kA machine-learning compiler that builds and runs LLMs across browsers, phones, and desktops using TVM-based code generation.mlc-ai/mlc-llm ↗details →
KTransformers17.3kA framework for running large Mixture-of-Experts models locally by splitting work between CPU and GPU to fit limited VRAM.kvcache-ai/ktransformers ↗details →
ggml14.8kThe low-level C tensor library that powers llama.cpp and whisper.cpp for running models efficiently on local hardware.ggml-org/ggml ↗details →
mistral-inference10.8kMistral AI's minimal official library for running Mistral open-weight models, with CLI demo and chat commands plus a Python API for text and multimodal inference.mistralai/mistral-inference ↗details →
PowerInfer9.6kA local inference engine that speeds up LLM serving on consumer GPUs by keeping frequently used neurons on the GPU.Tiiny-AI/PowerInfer ↗details →
Nexa SDK8.1kAn on-device inference framework that runs LLMs and multimodal models across CPU, GPU, and NPU on desktop and mobile.qualcomm/nexa-sdk ↗details →
Open-source and commercial AI tools, grouped by what they do — filter by access (open source, freemium, commercial, enterprise) with the chips above. Open-source projects link to a detail page with a plain-English overview and a getting-started guide; their star counts are pulled live from GitHub and refreshed every few hours.