Tag
vLLM serves language models with PagedAttention, continuous batching and an OpenAI-compatible API. In October 2026 it is on 0.32.0, V1 has been the only engine since 0.11 and prefix caching is on by default. This guide covers starting it with vllm serve, quantization by hardware, metrics and when to pick something else.
SGLang adds a Python DSL for controlling LLM generation with constrained decoding, parallel branching, and RadixAttention, the structure that indexes the KV cache as a radix trie to reuse shared prefixes across requests. When that pattern exists, speedups over vLLM reach up to 5 times; without it, the advantage shrinks.
Text Generation Inference (TGI) is the Hugging Face stack for serving open LLMs in production: continuous batching, 4-bit and 8-bit quantization, streaming, and an OpenAI-compatible API. After a brief restrictive-licence episode in 2023, it returned to Apache 2.0 in version 2.0.
