Guidance

// Overview

Guidance is a Python library from Microsoft for steering language models. Instead of writing one big prompt and hoping the model returns the right format, you build the prompt and the generation together in code, mixing fixed text with gen() and select() calls that produce model output.

It is aimed at developers who need the output to follow a specific shape, such as a number, a choice from a list, or a string that matches a regular expression or context-free grammar. Because the constraints are applied during generation, you get valid output without extra retries or post-processing.

As a structured-output and orchestration tool, Guidance works across several backends, including Transformers, llama.cpp, and OpenAI. You can also interleave control logic such as conditionals, loops, and tool use directly with generation.

// What it does

• Pythonic interface where model objects are immutable and you append prompt and generation with the += operator
• Constrained generation with regular expressions and context-free grammars to guarantee output syntax
• select() to force the model to pick from a known list of options
• Roles via system(), user(), and assistant() context managers
• Reusable @guidance functions that compose like the built-in gen() and select()
• Offline grammar debugging with grammar.match() and the Mock model, with no API calls

// Getting started

Install Guidance from PyPI, then build a chat-style prompt and capture constrained output. You also need a backend such as Transformers for the model you want to run.

pip install guidance

from guidance import system, user, assistant, gen
from guidance.models import Transformers

phi_lm = Transformers("microsoft/Phi-4-mini-instruct")
lm = phi_lm

with system():
    lm += "You are a helpful assistant"

with user():
    lm += "Hello. What is your name?"

with assistant():
    lm += gen(name="lm_response", max_tokens=20)

print(lm['lm_response'])

from guidance import select

lm = phi_lm

with user():
    lm += "What is the capital of Sweden? Answer A, B, C, or D."

with assistant():
    lm += select(["A", "B", "C", "D"], name="model_selection")

print(lm['model_selection'])

from guidance import gen
from guidance.models import Mock

grammar = "expr=" + gen(regex=r"\d+([+*]\d+)*", name="expr")

assert grammar.match("expr=12+7*3") is not None
assert grammar.match("expr=12+*3") is None

lm = Mock(b"<s>expr=12+7*3")
lm += grammar
print(lm["expr"])  # 12+7*3

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Forcing model output into a strict format such as JSON, a single number, or a value matching a regex
• Building multiple-choice or classification flows where the answer must come from a known list
• Interleaving control flow like loops and conditionals with generation, for example answering many questions in one pass
• Iterating on and testing constraint grammars offline before spending tokens on real model calls

// How Guidance compares

Guidance alongside other open-source structured output tools AI/TLDR tracks, ranked by GitHub stars.