Ollaya · 2026-09-24 · notable
Ollaya — an Ollama-style runtime for local decision models
Ollaya is an Apache-2.0 Rust tool that pulls and serves open decision models such as Laya, decider, Kev and Qwen3Guard on your own machine, behind a TypeSafe-compatible API. It reached 266 points on Hacker News.
Ollaya does for typed decision models what Ollama does for LLMs: one binary to pull, run and serve them locally.
Key specs
| Laya latency (rtx 4090) | 8–10 ms |
|---|---|
| Hn points | 266 |
What is it?
Ollaya is an open-source runtime for decision models, the small models that take text or JSON plus a typed question and return calibrated probabilities instead of generated text. Its CLI mirrors Ollama's commands (pull, run, serve, list, ps, rm), and it ships Laya, decider, NLI, GLiClass, Qwen3Guard, Kev and Von in its library.
How does it work?
Under the hood, each model is a small ONNX graph of about 3 MB that points at the original weights in the author's Hugging Face repo instead of re-hosting them, and Ollaya runs it on ONNX Runtime on CPU or CUDA. The daemon listens on localhost:11435 and exposes /v1/decisions, which the README calls wire-identical to TypeSafe's API, plus a native /api/decide endpoint with timings.
Why does it matter?
Teams that call a hosted decision API for routing or triage can point the same requests at their own hardware. The Ollaya site lists 8–10 ms for Laya on an RTX 4090 against 236–276 ms for hosted TypeSafe Jev, with no per-token fees and the data never leaving the machine.
Who is it for?
developers building agent routing and triage
Try it
curl -fsSL https://ollaya.dev/install.sh | sh && ollaya run laya