Ollaya

// Overview

Ollaya is an open-source runtime for decision models: small models that take a piece of text or JSON state plus a typed question and return calibrated probabilities for each allowed answer, without generating text. Its stated goal is to run open decision models locally the way Ollama runs LLMs, so the command line mirrors Ollama's verbs: serve, run, pull, list, ps, show, rm, cp, stop and create.

Each model in the Ollaya library is a small ONNX graph of about 3 MB that references the original weights in the author's Hugging Face repository instead of re-hosting them. Inference runs natively on ONNX Runtime, on CPU or NVIDIA CUDA. The library covers Laya (English and multilingual routers), decider (built on Qwen3.5), NLI for zero-shot classification, GLiClass, Qwen3Guard as a safety guard, Jared Palmer's Kev and the ModernBERT-based Von.

The daemon exposes a TypeSafe-compatible API, so code written against TypeSafe's hosted decision service can point at localhost instead, plus native endpoints with timing data. It also ships an MCP server and a desktop app for macOS, Windows and Linux. The project is written in Rust and split into crates for the CLI and daemon, HTTP server, API types, model registry, decision logic, ONNX runner and language detection.

// What it does

• One binary for the daemon and an Ollama-style CLI (serve, run, pull, list, ps, show, rm, cp, stop, create)
• TypeSafe-compatible POST /v1/decisions and /v1/systemone endpoints, plus native /api/decide with timings
• Native ONNX Runtime inference on CPU or CUDA, with model graphs that point at the authors' own Hugging Face weights
• Built-in library including laya, decider, nli, gliclass, qwen3guard, kev and von
• Presets such as triage and an agent preset that rates a command an AI agent is about to run for risk and destructiveness
• Custom models through Modelfiles that set question sets and precision
• MCP server for clients such as Claude Code, Claude Desktop and Cursor
• Desktop app for macOS, Windows and Linux, and Docker images with CUDA

// Getting started

Ollaya installs as a single binary on Linux (x86_64 and arm64), macOS on Apple silicon and Windows x64 (CPU). For NVIDIA GPUs on Windows, use WSL 2 with the Linux installer. The server listens on localhost:11435.

curl -fsSL https://ollaya.dev/install.sh | sh

irm https://ollaya.dev/install.ps1 | iex

docker run -d --gpus=all -p 11435:11435 ghcr.io/ollaya-dev/ollaya:cuda

ollaya run laya --preset triage "I was charged twice for my subscription..."
ollaya pull kev

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Routing and triage of support messages or agent steps with millisecond answers on local hardware
• Replacing calls to a hosted TypeSafe decision API with a local, fee-free endpoint
• Checking a command an AI coding agent wants to run before it executes
• Running safety classification with Qwen3Guard across many languages without sending data to a cloud API

// How Ollaya compares

Ollaya alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.