# Ollaya

> Pull, run and serve open decision models locally, the way Ollama runs LLMs

- Category: Models & Inference / Local Runtimes
- Language: Rust
- Licence: Apache-2.0
- Repository: https://github.com/ollaya-dev/ollaya
- Website: https://ollaya.dev/
- Canonical page: https://ai-tldr.dev/tools/ollaya/
- Updated: 2026-09-24

## In short

Ollaya is an open-source runtime for decision models: small models that take a piece of text or JSON state plus a typed question and return calibrated probabilities for each allowed answer, without generating text. It sits in the Local Runtimes category of the Models & Inference tools and is open-source, written in Rust and licensed Apache-2.0.

## Overview

Ollaya is an open-source runtime for decision models: small models that take a piece of text or JSON state plus a typed question and return calibrated probabilities for each allowed answer, without generating text. Its stated goal is to run open decision models locally the way Ollama runs LLMs, so the command line mirrors Ollama's verbs: serve, run, pull, list, ps, show, rm, cp, stop and create.

Each model in the Ollaya library is a small ONNX graph of about 3 MB that references the original weights in the author's Hugging Face repository instead of re-hosting them. Inference runs natively on ONNX Runtime, on CPU or NVIDIA CUDA. The library covers Laya (English and multilingual routers), decider (built on Qwen3.5), NLI for zero-shot classification, GLiClass, Qwen3Guard as a safety guard, Jared Palmer's Kev and the ModernBERT-based Von.

The daemon exposes a TypeSafe-compatible API, so code written against TypeSafe's hosted decision service can point at localhost instead, plus native endpoints with timing data. It also ships an MCP server and a desktop app for macOS, Windows and Linux. The project is written in Rust and split into crates for the CLI and daemon, HTTP server, API types, model registry, decision logic, ONNX runner and language detection.

## What it does

- One binary for the daemon and an Ollama-style CLI (serve, run, pull, list, ps, show, rm, cp, stop, create)
- TypeSafe-compatible POST /v1/decisions and /v1/systemone endpoints, plus native /api/decide with timings
- Native ONNX Runtime inference on CPU or CUDA, with model graphs that point at the authors' own Hugging Face weights
- Built-in library including laya, decider, nli, gliclass, qwen3guard, kev and von
- Presets such as triage and an agent preset that rates a command an AI agent is about to run for risk and destructiveness
- Custom models through Modelfiles that set question sets and precision
- MCP server for clients such as Claude Code, Claude Desktop and Cursor
- Desktop app for macOS, Windows and Linux, and Docker images with CUDA

## Getting started

Ollaya installs as a single binary on Linux (x86_64 and arm64), macOS on Apple silicon and Windows x64 (CPU). For NVIDIA GPUs on Windows, use WSL 2 with the Linux installer. The server listens on localhost:11435.

1. Install on Linux or macOS

   The install script sets up the ollaya binary.

```bash
curl -fsSL https://ollaya.dev/install.sh | sh
```

2. Install on Windows

   The PowerShell installer needs no administrator rights.

```powershell
irm https://ollaya.dev/install.ps1 | iex
```

3. Or run it in Docker with a GPU

```bash
docker run -d --gpus=all -p 11435:11435 ghcr.io/ollaya-dev/ollaya:cuda
```

4. Ask a typed question

   Run a model with a preset against a piece of text. Other models are fetched with ollaya pull.

```bash
ollaya run laya --preset triage "I was charged twice for my subscription..."
ollaya pull kev
```

## When to use it

- Routing and triage of support messages or agent steps with millisecond answers on local hardware
- Replacing calls to a hosted TypeSafe decision API with a local, fee-free endpoint
- Checking a command an AI coding agent wants to run before it executes
- Running safety classification with Qwen3Guard across many languages without sending data to a cloud API

## Version history

- 2026-09-24: Ollaya shipped its first releases and launched on Hacker News as an Ollama-style runtime for decision models. v0.6.0 on September 25 added the Von, Kev and Qwen3Guard model families, an agent review preset and a redesigned macOS menu bar.