Architecture
vLLM: Serving LLMs in Production with Very High Throughput
vLLM serves language models with PagedAttention, continuous batching and an OpenAI-compatible API. In October 2026 it is on 0.32.0, V1 has been the only engine since 0.11 and prefix caching is on by default. This guide covers starting it with vllm serve, quantization by hardware, metrics and when to pick something else.