developer-roadmap/roadmaps/inference-engineering/content/constrained-decoding@E8mQE9mlIPW6wpmJBQe0b.md

Constrained Decoding

Constrained decoding is the mechanism that enforces structured outputs at the token level. At each sampling step, logits for tokens that would violate the active schema or grammar are masked to zero before sampling, so only valid continuations can be selected. Grammar compilation adds overhead to time to first token, and the interaction with speculative decoding is important to note: some engines disable speculation under hard constraints because draft token acceptance rates drop sharply when only a narrow set of tokens is valid at each step. .

Visit the following resources to learn more:

• @opensource@Guidance: guided generation
• @opensource@Outlines: structured generation
• @article@A Guide to Structured Generation Using Constrained Decoding
• @article@Constrained decoding definition