# Atomic Chat

> A local AI app with its own inference engines that runs open-weight LLMs offline and serves them over an OpenAI-compatible API

- Category: Models & Inference / Local Runtimes
- Language: TypeScript
- Licence: Apache-2.0
- Repository: https://github.com/AtomicBot-ai/Atomic-Chat
- Website: https://atomic.chat
- Canonical page: https://ai-tldr.dev/tools/atomic-chat/
- Updated: 2026-09-21

## In short

Atomic Chat is an open-source app for running open-weight large language models on your own computer. You download a model such as Llama, Gemma, Qwen, Mistral or Phi from Hugging Face, and chat with it privately and offline in a ChatGPT-style window. It sits in the Local Runtimes category of the Models & Inference tools and is open-source, written in TypeScript and licensed Apache-2.0.

## Overview

Atomic Chat is an open-source app for running open-weight large language models on your own computer. You download a model such as Llama, Gemma, Qwen, Mistral or Phi from Hugging Face, and chat with it privately and offline in a ChatGPT-style window. Desktop builds exist for macOS, Windows and Linux, and there are iOS and Android apps.

Under the hood it ships three inference engines: the upstream llama.cpp build, the team's own llama.cpp fork with TurboQuant KV-cache compression, and MLX-VLM for Apple Silicon. It adds speed-ups such as Multi-Token Prediction and DFlash speculative decoding on supported models, plus a Flash Attention switch.

Every engine sits behind one OpenAI-compatible server at localhost:1337, so coding agents, CLIs and IDE plugins that speak the OpenAI API can use your local models by changing only their base URL. The app can also launch agents such as Claude Code, Codex CLI, Cline, OpenCode and Goose from its Integrations tab, connect MCP servers, and mix in cloud providers like OpenAI, Anthropic and Mistral with your own keys.

Atomic Chat began as a fork of Jan by Menlo Research and has since grown its own engines and roadmap. It is licensed under Apache 2.0.

## What it does

- Runs open-weight LLMs from Hugging Face (Llama, Gemma, Qwen, Mistral, Phi) fully on your machine
- Three engines behind one API: upstream llama.cpp, a TurboQuant llama.cpp fork with a smaller KV cache, and MLX-VLM on Apple Silicon
- Speculative decoding (Multi-Token Prediction, DFlash, EAGLE-3 on MLX) and a Flash Attention toggle for faster generation
- OpenAI-compatible server at localhost:1337, loopback-only by default, with an option to expose it on your LAN
- One-click launch of coding agents (Claude Code, Codex CLI, Cline, OpenCode, Goose and others), MCP servers, artifacts preview and custom assistants
- Optional cloud providers (OpenAI, Anthropic, Mistral, Groq and more) with your own key, switchable per chat

## Getting started

Most people install the prebuilt app; developers can build it from source. The commands below come from the project README.

1. Install the app

   Download the macOS universal .dmg, the Windows x64 installer or the Linux AppImage from GitHub Releases or atomic.chat. Mobile builds are on the App Store and Google Play. On Linux the AppImage runs without an installer or root.

2. Run on Linux

   Make the AppImage executable and start it. If it asks about FUSE, install fuse and libfuse2 (Debian/Ubuntu). Only GGUF models run on Linux; Vulkan GPU acceleration is detected on first launch.

```bash
chmod +x Atomic.Chat_*_amd64.AppImage
./Atomic.Chat_*_amd64.AppImage
```

3. Download a model and chat

   Inside the app, pick an open-weight model from Hugging Face, download it and start chatting. The README suggests 8 GB of RAM for 3B models, 16 GB for 7B and 32 GB for 13B.

4. Call it from code

   With a model loaded, any OpenAI client can talk to the local server; only the base URL changes.

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:1337/v1", api_key="not-needed")

resp = client.chat.completions.create(
    model="<model-id-loaded-in-atomic-chat>",
    messages=[{"role": "user", "content": "Say hello in one word"}],
)
print(resp.choices[0].message.content)
```

5. Build from source (optional)

   Needs Node.js 20+, Yarn 4.5.3+, Make and Rust (for Tauri). make dev installs dependencies, builds the core and launches the app.

```bash
git clone https://github.com/AtomicBot-ai/Atomic-Chat
cd Atomic-Chat
make dev
```

## When to use it

- Chat with open-weight models privately and offline on a laptop or phone
- Point coding agents such as OpenCode, Goose or Kilo Code at local models through the localhost:1337 OpenAI-compatible endpoint
- Get faster, lower-memory local inference on Apple Silicon or consumer GPUs with MLX, TurboQuant KV cache and speculative decoding
- Mix local models and cloud providers in one app, switching per conversation

## Version history

- 2026-09-21 v2.0.44: Added image generation through stable-diffusion.cpp and remote access to local models from other devices over LAN or a Cloudflare tunnel. It also added one-click launch of Z.ai's ZCode and a reasoning-effort setting for cloud models.
