Laya

// Overview

Laya is an open-weight decision model that answers typed questions about a piece of text without writing any text back. You hand it a state — an email, a ticket, a JSON document — and a set of questions, and it returns a probability distribution per question plus a confidence number. Because nothing is generated token by token, there is no JSON to repair and no parsing step, and the published median latency for a single query is 32.8 milliseconds.

Convai Innovations publishes three Apache-2.0 checkpoints. The base `laya` checkpoint is a 421M ModernBERT-large model with 512 tokens of context for English. `laya-multilingual` is a 322M mmBERT-base model with 1024 tokens of context covering more than 100 languages. `laya-typed-decisions` is the base checkpoint fine-tuned on four synthetic workflows — agent-trace observability, customer service, invoice processing and security incidents — and scores 0.766 on the typed-decisions benchmark against TypeSafe Jev 1.13.0's 0.727. Training uses RLCD, where the reward is a strictly proper scoring rule, so expected reward is maximised only by reporting honest probabilities.

Laya sits in the structured-output layer alongside SemIf and the constrained-decoding libraries such as Outlines and XGrammar. The difference is the same one SemIf draws: those tools shape the tokens a generative model emits, while Laya replaces generation entirely for the narrow case of scoring a fixed set of described options. Convai Innovations is explicit about the limit — zero-shot accuracy on typed-decisions is 0.362 for the base checkpoint and 0.352 for the multilingual one, so the project describes itself as a fast base to specialise rather than a ready-made zero-shot decision engine.

// What it does

• Three question primitives: `choice` picks an option with per-option probabilities, `score` places a value on an ordinal rubric, and `noul` returns a calibrated P(true)
• All questions in a request are answered in a single forward pass, with no output tokens sampled
• A confidence number per answer, so low-confidence results can be escalated to a human instead of acted on
• A Router that lazily picks the English, multilingual or typed-decisions checkpoint per request and explains its choice
• Built-in question presets for model routing, prompt guardrails, content moderation and support triage
• Memory controls on the Router: preload chosen checkpoints, cap how many stay resident, or unload them all

// Getting started

Laya installs from PyPI and pulls its weights from the Hugging Face Hub on first use. All three checkpoints are Apache-2.0.

pip install laya

import laya

agent = laya.load("convaiinnovations/laya")

state = {
    "from": "user@acme.com",
    "subject": "Duplicate charge on invoice #4411",
    "body": "Hi, we were billed twice for March.",
}

questions = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this email?",
        "criteria": {
            "billing": "invoices, payments, refunds",
            "technical": "bugs, outages, system errors",
            "sales": "pricing, new contracts",
            "other": "everything else",
        },
    },
    "churn_risk": {
        "type": "noul",
        "instructions": "Does the user threaten to cancel or leave?",
    },
}

result = agent.predict(state, questions)
answers = result["answers"]

dept = answers["department"]["choice"]
conf = answers["department"]["confidence"]

if conf >= 0.85:
    route_automatically(dept)
else:
    escalate_to_human_agent(dept, reason=f"Low confidence ({conf:.2f})")

from laya import Router

router = Router()
router.predict({"body": "I was charged twice, please refund."}, questions)
router.predict(state, questions, model="typed-decisions")
router.predict(state, questions, lang="de")

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Routing, triage and moderation inside an agent loop, where a full generated answer is wasted work
• Classifying support tickets or invoices with a confidence threshold that decides what a human still sees
• Prompt guardrails and content safety checks that have to run on every request without adding an LLM call
• Scoring multilingual text when the English checkpoint cannot read the script, via the Router's automatic fallback

// How Laya compares

Laya alongside other open-source structured output tools AI/TLDR tracks, ranked by GitHub stars.