Artificial Intelligence
Hugging Face TGI: Serving Open Models at Scale
Text Generation Inference (TGI) is the Hugging Face stack for serving open LLMs in production: continuous batching, 4-bit and 8-bit quantization, streaming, and an OpenAI-compatible API. After a brief restrictive-licence episode in 2023, it returned to Apache 2.0 in version 2.0.