Tag
Using an LLM to judge another LLM became widespread in 2024 and remains, in 2026, the only scalable way to evaluate qualitative quality in LLM systems. It is reliable when judge-human correlation exceeds 0.7 on 30 cases and gets recalibrated quarterly; below that threshold, do not trust the number.
After a decade of Prometheus, three years of consolidation around OpenTelemetry, and the open stack now mature with Grafana, Loki, and Tempo, concrete recommendations for teams starting or reviewing their observability layer: what fits, what is excess, and what to avoid.
Agents that chain calls to models, tools and memory are hard to debug without instrumentation designed for them. After a long year running agents in production, I cover what to measure first, which standards are consolidating, and which costly mistakes are avoided by getting the traces right from the start.
Product-market fit for LLM-powered products still depends on the same classic signals: cohort retention, NPS, and revenue expansion. What changes are the higher quality baseline, faster competitor iteration, and where durable moats come from: proprietary data, workflow integration, and network effects.
OpenTelemetry is the CNCF project, graduated in May 2026, that unifies logs, metrics, and traces under one SDK and the OTLP protocol, without locking you into a single backend. Traces have been stable since 2021 and metrics since 2023; logs are still maturing, but already worth adopting on new projects.
Useful startup metrics are the data points that let you evaluate a company's real performance: general performance metrics (MRR, retention, churn), area KPIs (CAC, LTV, NPS), and the AARRR framework, always prioritising retention and the LTV:CAC ratio over vanity metrics.
