Constrained generation with Outlines

Outlines is a Python library for constrained generation: instead of asking an LLM to return JSON and crossing your fingers, it forces the model to emit only tokens that keep the output inside a schema, so the result is always valid by construction. In this guide you will see which formats it supports (JSON, regular expressions and grammars), how it guarantees correct output by masking logits, how to use it with vLLM and models you run on your own machine, and how it differs from Instructor. The same explanation is available in Spanish.

Key takeaways

• Outlines is the reference library from the company dottxt for constrained generation; it has over 14,000 stars on GitHub and 65 million downloads, and its latest version is v1.3.1, released on 30 June 2026.
• The guarantee is mathematical, not statistical: at each step the tokens that would break the format are set to zero, so a JSON with an unclosed brace is simply never a reachable option.
• You can constrain output to a Pydantic model, a JSON Schema, a regular expression or a context-free grammar in Lark notation; also to a choice among several options with Literal.
• The engine lives in outlines-core, rewritten in Rust for speed and portability; it precompiles an index once and reuses it on every request, so the per-token cost is minimal.
• It works with 12 backends (vLLM, Transformers, Ollama, llama.cpp, MLX, SGLang, TGI and the OpenAI, Anthropic or Gemini APIs), but the real by-construction guarantee only exists when you control inference on your own machine.

What is constrained generation?

Constrained generation (also called guided decoding) means limiting, at each generation step, the set of tokens the model may choose so that the resulting sequence always meets a target format. At each step an LLM produces a probability distribution over its whole vocabulary, some tens of thousands of tokens. Without any constraint it samples from that distribution as is, and an extra comma, an unclosed brace or a chatty preamble like "here is the JSON" can slip in.

Outlines steps in right there. It takes your format specification and, before sampling, strikes out (sets to zero probability) every token that could not extend a valid string. The model only chooses from what remains. Compared with the general technique of constrained decoding, which we already covered as a concept, this article focuses on the concrete library: its API, its engine and its practical use.

What does Outlines support: JSON, regex and grammars?

The strength of Outlines is the variety of constraints it accepts through the same call. Since the v1.0 rewrite, released on 18 June 2025, the interface was simplified: you build a model with a factory function and call it, passing the output type you want. A Pydantic model is the most common form:

import outlines
from transformers import AutoModelForCausalLM, AutoTokenizer
from pydantic import BaseModel

class Invoice(BaseModel):
    number: str
    total: float
    lines: list[str]

name = "HuggingFaceTB/SmolLM2-1.7B-Instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(name),
    AutoTokenizer.from_pretrained(name),
)

# The second argument is the output type: here, a Pydantic schema
invoice = model("Extract the data from: Invoice A-0012, total 128.50", Invoice)
print(invoice)

But the output type need not be an object. You can pass a regular expression to force, say, a postal code or a date; a Literal to force a choice among a few labels; or a context-free grammar in Lark notation when the structure is more complex than plain JSON:

from typing import Literal

# A regular expression: exactly a code of the form A-0000
code = model("Give me a valid invoice code", r"[A-Z]-\d{4}")

# A closed choice among labels
sentiment = model("Classify: the delivery arrived late", Literal["positive", "negative", "neutral"])

This combination (Pydantic, JSON Schema, regex, grammars and closed choices) covers almost any structured extraction or classification need.

How does Outlines guarantee valid output?

Here is the difference from the classic retry loop. Outlines does not generate and then validate: it constrains during generation. Its engine, the outlines-core library rewritten in Rust, builds from your schema an index that maps each state of a finite-state machine to the set of vocabulary tokens that are legal in that state. That idea comes from the library’s founding paper, by Brandon T. Willard and Rémi Louf, which reformulates guided generation using finite-state machines and states that the method "adds little overhead to the token sequence generation process".

The flow at each step is always the same:

1. The model produces the logits (the scores for each vocabulary token).
2. Outlines looks up the index for the current state and masks every token that would not keep the output inside the schema.
3. It normalises and samples only over the legal subset.
4. The automaton advances to the next state and repeats.

The result is a guarantee by construction: there is no branch of the generation tree that can produce broken JSON, because the token that would break it is unreachable. The index is compiled once per schema and reused on every request, so the marginal cost per token is very low. One honest limitation is worth remembering: the guarantee is syntactic, not semantic. The JSON will be valid, but if the model misread the document, the values will still be wrong.

How do you use Outlines with vLLM and local models?

Where Outlines really shines is on an inference engine you control. It integrates with vLLM both in offline mode (with no server to stand up) and through the server, where it appears as one of the guided-decoding backends. In offline mode the call is direct:

import outlines
from vllm import LLM

# Install vLLM BEFORE Outlines to avoid a version conflict
model = outlines.from_vllm_offline(LLM("Qwen/Qwen3-4B-Instruct"))
invoice = model("Extract the data from the attached invoice...", Invoice)

The same pattern works for from_llamacpp, from_mlxlm or from_ollama: you pick the factory function for the runtime you already use and the rest of the code does not change. Because the constraint is applied over the logits, any open model works, with no need for it to be fine-tuned for tool calling. If you are deciding which open model to run, the comparison of open models with tool calling will help you choose. Practical requirement: Outlines needs Python 3.10 or higher (up to 3.13).

Outlines or Instructor?

The two libraries solve the same pain (getting reliable structured outputs) with opposite philosophies, and the choice depends on where your model runs.

In short: if you serve an open model with vLLM, llama.cpp or Transformers, Outlines gives you the strongest guarantee because it acts at the token level. If you work against a closed API with no access to the inference engine, Instructor is the pragmatic route, since it wraps the client and retries until the response validates. Outlines also supports API models (with from_openai), but in that case it defers to the provider’s structured mode and loses token masking.

Frequently asked questions

Does constrained generation slow the model down a lot?

Not much. The automaton index is compiled once per schema, and then per-token masking is a very cheap operation, which is why the original paper speaks of "little overhead". In practice each token can be slightly slower than without a constraint, but full retry passes disappear, so at large-scale extraction the maths works out in your favour.

Does Outlines improve the quality of the model’s answers?

Not in the semantic sense. It guarantees that the output meets the format (that the JSON parses and respects the schema), but not that the values are correct. If the model misreads the input document, it will produce a perfectly valid JSON with wrong data. The constraint fixes the format, not the understanding.

Can I use Outlines with OpenAI or Anthropic models?

Yes, with from_openai or from_anthropic, but note the nuance: with a closed API Outlines cannot mask the logits, so it relies on the provider’s own structured-output mode. The formal by-construction guarantee only exists when you control inference with vLLM, Transformers or llama.cpp on your own machine.

Conclusion

Outlines moves format validation from your application code down into the generation layer itself, which is where the marginal cost is lowest and the guarantee strongest. For large-scale structured extraction, agents that emit tool calls or synthetic-data generation with a fixed schema, it turns a probabilistic problem into a certainty by construction. The next step is simple: install the library with pip install outlines, pick an open model you serve with vLLM, define your schema with Pydantic and compare the result with what you patch today with retries.

Sources: [1] Official Outlines documentation^[1], [2] Outlines on GitHub^[2], [3] Efficient Guided Generation for Large Language Models, by Willard and Louf^[3], [4] The outlines-core 0.1.0 launch on the Hugging Face blog^[4], [5] The outlines package on PyPI^[5].

Sources

1. Official Outlines documentation
2. Outlines on GitHub
3. Efficient Guided Generation for Large Language Models, by Willard and Louf
4. The outlines-core 0.1.0 launch on the Hugging Face blog
5. The outlines package on PyPI

Route: Self-hosted Agentic Models and Tool Calling