DPO and alternatives to RLHF: practical state in 2026

Direct Preference Optimization (DPO) and its variants, IPO, KTO, and SimPO, have displaced RLHF as the preferred alignment method for language models: they drop the separate reward model, cut training cost, and are easier to reproduce. RLHF still has an edge only for frontier models with very large budgets.

April 28, 2026 3 min 1.2K 4.6

Artificial Intelligence

Alignment evaluation: RLHF, DPO, and recent alternatives

Three years after RLHF became popular, the model-alignment field is far richer. A review of RLHF, DPO, and newer methods such as KTO and ORPO, with criteria for choosing between them.

February 8, 2025 5 min 429