DSPy Optimizers Explained: How Prompts Get Compiled

// In plain English

In DSPy you write a small program — a signature that says what goes in and out, and a module like dspy.ChainOfThought that picks a reasoning strategy. But that program ships with a bland default prompt. An optimizer is the part that takes that bland program and rewrites the prompt for you so the model actually scores well on your task. In DSPy's older docs you'll see these called teleprompters — same thing.

The everyday analogy: think of a new hire who technically knows the job but has never seen your tickets. You don't rewrite their brain. Instead you hand them a folder of solved examples — "here's a tricky billing case and how we labeled it" — and a clear rubric for what counts as a good answer. After studying those, they get noticeably better. A DSPy optimizer does exactly that: it studies labeled examples against a scoring rule and bakes the best examples and instructions into the prompt. You supply the folder (a trainset) and the rubric (a metric); the optimizer does the studying.

The verb DSPy uses for this is compile. You call optimizer.compile(program, trainset=...) and get back the same program carrying a tuned prompt under the hood. Nothing about your Python changed — only the hidden prompt strings the model sees did. That's why the DSPy pitch is "programming, not prompting": you keep writing code, and compilation handles the prompt text.

// Why it matters

Most people find DSPy signatures and modules easy to grasp and then get stuck on the optimizers — they feel like magic. Understanding what the compiler is really doing is what turns DSPy from a curiosity into something you can ship. The optimizer is where the actual quality gains come from; everything else is plumbing.

• It replaces few-shot guesswork with search. Hand-picking which examples to paste into a prompt is slow and biased toward the cases you happen to remember. An optimizer tries many combinations and keeps the ones that measurably help — it searches a space you'd never explore by hand.
• It survives model swaps. A prompt hand-tuned for one model often degrades on the next. Because optimization is just compile against a metric, you point DSPy at the new model and re-run it. The search re-finds good few-shot demos and instructions for that model — no string rewriting.
• It makes 'better' a number, not a vibe. Optimization forces you to define a metric. Once you have one, you stop arguing about whether a prompt edit helped and start reading a score on held-out data. That single discipline is worth more than any prompt trick.

Who should care? Anyone who has built a working DSPy program and wants it to actually be good: people running classifiers, extraction, RAG pipelines, or multi-step agents where accuracy is measurable. If you only need a one-off chatbot reply, you can skip optimization entirely — modules run fine uncompiled. The optimizer earns its keep when correctness matters and you can score it.

// How the compile loop works

Every DSPy optimizer needs the same three ingredients, and the compile step is a loop over them. Get these three right and the rest is choosing which optimizer to run.

• A program — your module(s), e.g. a ChainOfThought wrapping a signature. This is the thing being tuned.
• A trainset — a list of dspy.Example objects: realistic inputs, ideally with the correct outputs (the gold labels). This is what the optimizer learns from.
• A metric — a Python function metric(example, prediction) that returns a number (often 1.0 for correct, 0.0 for wrong, or a partial score). This is the rubric the optimizer maximizes.

Here's the part that surprises people: most optimizers generate the few-shot examples themselves rather than just pasting your raw labels. The optimizer runs your program on trainset inputs, watches what the model produces, scores each run with your metric, and keeps the input→output traces that scored well. Those successful traces become the few-shot demonstrations embedded in the final prompt. This trick is called bootstrapping: the model's own good answers become its teaching examples.

After the loop, the optimizer assembles the winners — selected few-shot demos plus (for some optimizers) tuned instruction text — into the program's prompt and hands you back the compiled module. The diagram below shows the whole journey from a raw program to a tuned one.

import dspy

# A program: chain-of-thought over a simple signature.
classify = dspy.ChainOfThought("message -> category")

# A trainset: realistic inputs with gold labels.
trainset = [
    dspy.Example(message="I was double charged", category="billing").with_inputs("message"),
    dspy.Example(message="The app keeps crashing", category="technical").with_inputs("message"),
    # ...ideally 50-300 varied examples...
]

# A metric: 1.0 if the predicted category matches the gold label.
def accuracy(example, prediction, trace=None):
    return float(example.category == prediction.category)

# Compile = run the optimizer's search loop over the trainset.
optimizer = dspy.BootstrapFewShot(metric=accuracy)
tuned = optimizer.compile(classify, trainset=trainset)

# `tuned` is the same program, now carrying optimized few-shot demos.
print(tuned(message="Why is my invoice wrong?").category)

// BootstrapFewShot vs MIPRO

DSPy ships several optimizers; two names dominate the discussion. They sit at different points on the cheap-and-simple to powerful-and-expensive spectrum, and knowing which to reach for first saves a lot of wasted compute.

BootstrapFewShot — tune the examples

BootstrapFewShot does exactly the bootstrapping loop above: it runs your program, collects traces that pass the metric, and selects a handful as few-shot demonstrations. It mostly tunes which examples go in the prompt — it leaves the instruction wording roughly as written. It's cheap, fast, and a great first thing to try. A common stronger variant, BootstrapFewShotWithRandomSearch, generates several candidate demo sets and keeps the best-scoring one on a validation split.

MIPRO — tune the instructions too

MIPROv2 (Multiprompt Instruction PRoposal Optimizer) goes further: it jointly optimizes both the instruction text and the few-shot examples. It uses an LLM to propose candidate instruction phrasings, pairs them with candidate demo sets, and runs a guided search (Bayesian-style) to find the combination that scores best — rather than trying everything blindly. That makes it more powerful on hard tasks, but it costs many more LLM calls to compile.

A sane default workflow: start with BootstrapFewShot, measure the lift on a held-out set, and only graduate to MIPROv2 if you need more and can afford the compile budget. DSPy also includes GEPA, a reflective optimizer that reads failures in natural language and proposes improved instructions through an evolutionary search — useful when instruction wording, not example choice, is the bottleneck.

// What good metrics and trainsets look like

An optimizer is only as good as what you feed it. It blindly maximizes your metric over your data — so a weak metric or a skewed trainset produces a confidently wrong prompt. This is where almost all real effort should go.

The metric

• For exact answers, use exact match. Classification or extraction with a known correct label? Return 1.0 on a match, 0.0 otherwise. Simple and reliable.
• For partial credit, return a float. If an answer can be partly right (e.g. it got 3 of 4 fields), score it proportionally so the optimizer can tell better from worse, not just pass from fail.
• For open-ended text, consider an LLM judge. When there's no single right string, a metric can call another model to rate the answer — see LLM evals. This is powerful but adds cost and noise, so validate the judge itself.
• Match the metric to the real goal. If you optimize for exact-string equality but users only care about meaning, you'll tune toward the wrong target. The metric is the objective — choose it as carefully as the prompt you're trying to avoid writing.

The trainset

• Realistic and varied beats large and clean. Include the messy, ambiguous, edge-case inputs you actually see in production — not just the easy ones. A prompt tuned only on easy cases aces the demo and flops live.
• A few dozen is often enough to start. BootstrapFewShot can show gains with tens of examples; MIPROv2 and heavier searches benefit from more. You don't need a giant dataset to begin.
• Always hold out a separate eval set. Compile on the trainset, but report your score on examples the optimizer never saw. Otherwise you're measuring memorization, not quality — the same overfitting trap as any ML workflow.

// When optimization actually pays off

Compiling isn't free — it makes many LLM calls against your trainset, costing time and money up front. That investment pays back at inference time only under certain conditions. Use this table to decide before you spend the budget.

A practical rhythm: build with plain modules first to confirm the pipeline works, define a metric and a small eval set, measure the uncompiled baseline, then compile and check whether the lift justifies the cost. If you can't measure a lift, you haven't earned the right to optimize yet — go fix the metric or the data.

// Going deeper

Once the metric → trainset → compile loop clicks, here's the territory worth exploring next.

Optimizing multi-stage programs end to end. Because DSPy modules are plain Python objects, you nest them — a retrieval step feeds a chain-of-thought step feeds a formatting step. The remarkable part is that one optimizer can tune the whole pipeline against a single end-to-end metric, propagating credit through stages you could never hand-tune in isolation. This is the main reason DSPy treats prompts as a compilation target rather than text to edit.

From prompts to weights. Some DSPy optimizers can use your trainset to drive fine-tuning of the underlying model, compiling a program down into tuned weights for a smaller, cheaper model instead of (or alongside) a longer prompt. This blurs the line between prompt optimization and model training and is why DSPy bills itself as a framework for building systems, not just prompts.

Production realities. Compiled prompts can get long because they embed selected demonstrations — watch your context window and per-call cost. Optimization is also stochastic: results vary run to run, so save and version your compiled program (DSPy can serialize it) rather than recompiling on every deploy. Re-compile deliberately when the model, the data, or the task changes — not on a whim.

The honest open problem. Every optimizer is downstream of your metric, and turning "is this answer good?" into a reliable number is the same unsolved challenge at the heart of all LLM evaluation. DSPy makes the search automatic; it cannot make the objective correct. The durable lesson: spend your effort on the metric and the held-out data, and let the optimizer handle the prompt strings. If you're still choosing tools, compare DSPy against the rest of the lightweight, typed frameworks and the broader question of whether you need a framework at all.