Janas

// Overview

Janas is a family of AI tools written in C with no dependencies beyond the C library. Its first component, Janas-LLM, runs large mixture-of-experts (MoE) language models on ordinary computers, with or without a GPU. The weights stay on disk: the experts each token needs are read from an NVMe SSD, and the experts that are reused often stay in an in-memory expert cache. The project says this is how an 80-billion-parameter model runs on a laptop with 32 GB of RAM.

The engine fills free memory with its expert cache and reads the disk with O_DIRECT; a --cache option sets a hard memory limit. An integrated GPU reads prompts through Vulkan, the engine measures and picks its own thread counts, and drafts (speculative decoding) from the Qwen3.5, Qwen3.6 and Qwen3-Next prediction block or Gemma 4's drafting model raise the writing speed without changing the text. Models come pre-converted to its .jns format on Hugging Face, or can be converted from GGUF.

Around the engine sit a chat client with MCP tools, an OpenAI-style HTTP server that also does code completion at an editor cursor, an MCP server, a benchmark tool and a model downloader. The project is labelled experimental: its README says the API and command line change without notice, the engine is deliberately hard on the machine and may push the desktop into swap, and every published number comes from one Intel Core Ultra 9 laptop.

// What it does

• Streams MoE experts from an NVMe SSD and keeps the hot ones in an expert cache, so models larger than RAM run; --cache sets a hard memory limit
• Pure C with no dependencies beyond the C library; prebuilt packages for Linux x86-64 and Windows 10/11 x86-64
• Reads prompts on an integrated GPU through Vulkan (JANAS_GPU=0 turns it off) and tunes its own thread counts
• Drafts (speculative decoding) for Qwen3.5, Qwen3.6, Qwen3-Next and Gemma 4 models
• janas-server: an OpenAI-style HTTP API and browser chat page, with code completion through /v1/completions with a suffix and llama.cpp's /infill
• janas-chat with MCP tools, janas-mcp to expose a model as an MCP server, janas-bench and janas-get

// Getting started

Janas targets x86-64 machines with AVX2; the README says only one Debian 13 laptop has been tested. Run it on an idle machine — the engine uses all the memory it can.

curl -LO https://github.com/prabanta-dev/janas/releases/latest/download/janas-linux-x86_64.tar.gz
tar xf janas-linux-x86_64.tar.gz && cd janas
./janas-chat

.\janas-chat

sudo apt install build-essential
sudo apt install libvulkan-dev glslang-tools vulkan-tools   # optional, GPU

./build.sh
./build.sh release test

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Run an MoE model such as Qwen3-Next-80B-A3B on a laptop whose RAM is smaller than the model file
• Get faster local generation than llama.cpp on the same hardware for the MoE and small dense models the project benchmarks
• Serve a local model to an editor for code completion, or to other tools over an OpenAI-style API or MCP
• Skip it for production or untested hardware — it is experimental, measured on one machine, and can push the desktop into swap

// How Janas compares

Janas alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.