Tag
Using an LLM to judge another LLM became widespread in 2024 and remains, in 2026, the only scalable way to evaluate qualitative quality in LLM systems. It is reliable when judge-human correlation exceeds 0.7 on 30 cases and gets recalibrated quarterly; below that threshold, do not trust the number.
Claude Sonnet 4.6 is the default model for most 2026 production workloads: it covers 80% of traffic with quality indistinguishable from Opus 4.7 in blind tests, at roughly 60% of Opus per-token price. Opus is still needed for complex reasoning and agentic coding on large codebases.
Reliable agents come from measurement, not from better models or prompts. A production agent evaluation setup starts with a golden dataset of 30 to 200 cases, split roughly 60 percent normal usage, 30 percent edge cases and 10 percent adversarial, and it never uses the same model as both worker and judge.
Two years running AI-assisted code review in a real team leave a clear balance: AI catches mechanical oversights well and writes useful pull-request summaries, but it struggles with architectural judgment and produces many false positives on subtle bugs. The single decision that helped the most was not blocking merges on its automated comments.
