Tag
An AI agent is a program that uses a language model as its brain to decide for itself which steps to take toward a goal: it reasons, calls external tools, observes the result and repeats that loop until it is done. Unlike a chatbot, it does not just answer; it acts.
Tested May 2026 recipe: oMLX 0.3.8 on Mac M5 Max with 128 GB, TurboQuant at 3.5-bit, Qwen 3.6 35B-A3B model stack, Claude Code wiring and real benchmarks.
The idea of UI generated on the fly instead of pre-built reached production in 2025. After a year of real-world use, the balance is more nuanced than the initial enthusiasm suggested.
Direct Preference Optimization (DPO) and its variants, IPO, KTO, and SimPO, have displaced RLHF as the preferred alignment method for language models: they drop the separate reward model, cut training cost, and are easier to reproduce. RLHF still has an edge only for frontier models with very large budgets.
Synthetic data has moved from a precarious substitute for real data to a central component of modern model training: the most reliable pattern expands a real core of 500 examples with thousands of synthetic paraphrases, provided you validate diversity, correctness, and distribution, and keep at least 30% real data to avoid model collapse.
Hybrid RAG in 2026 combines dense and lexical search fused with RRF, cross-encoder reranking over the top-50 candidates, structure-aware chunking, and continuous evaluation with Ragas or TruLens. It is the pattern that survives in serious production systems three years after the initial embeddings boom.
The first invoice for a production agent usually runs double or triple the estimate. This article walks through five real levers, in priority order, caching, routing, context control, batching, and telemetry, to cut cost without touching perceived quality.
LLM red teaming has gone from an esoteric activity to a mandatory practice. With the OWASP Agentic Top 10 and the CSA Agentic AI Red Teaming Guide converging on shared vocabulary, this is the operational playbook any team deploying agents needs to have.
Prompt engineering has moved from viral tricks to a discipline with reproducible patterns: few-shot, chain-of-thought, and structured output with function calling. Teams treating prompts like code (versioned, tested, and monitored) get consistently better results than those who improvise.
FinOps for AI counts different units than classic cloud FinOps: tokens, calls, computed embeddings and GPU time, all of which scale nonlinearly with use. The costliest habit is sending everything to frontier models; 40 to 70 percent of those calls run on mid-tier models with no noticeable quality loss. Uncached RAG and self-recursing agents do the rest.
Knowledge graphs spent two decades waiting for their moment. With LLMs now bridging free text and formal ontology, and the GraphRAG pattern already mature, the technology is back in the spotlight. Time to look at why it finally fits and where it actually pays off.
Large language models have spent two years promising effortless documentation for code, APIs and architecture. After watching dozens of projects try it, clear patterns emerge for where it works and where it just becomes more debt.
Guardrails frameworks promise to filter language-model inputs and outputs to block data leaks, harmful content, or hallucinations. After evaluating four of the most popular ones in production, I cover what they actually do, what latency and billing cost they add, and when they pay off over simpler controls.
Agents that chain calls to models, tools and memory are hard to debug without instrumentation designed for them. After a long year running agents in production, I cover what to measure first, which standards are consolidating, and which costly mistakes are avoided by getting the traces right from the start.
A caching proxy in front of a language model can cut the token bill significantly, but it introduces subtle risks if the design is not careful. Which cache types work in production, where the usual traps sit, and how to add them without degrading the experience.
An inference router decides which model answers each incoming request, weighing cost, latency and how hard the request actually is. Well-built inference routers cut total token spend by 30 to 70 percent with no quality loss the user can perceive. Four patterns cover most cases: length, task type, an auxiliary classifier, and learned routing.
AI testing breaks the assumption every automated suite was built on, because the same input no longer produces the same output. Anthropic documents that temperature 0.0 is still not fully deterministic, and OpenAI's seed parameter only promises mostly reproducible results. What works instead is a layered belt of checks that catches regressions without tripping over ordinary variance.
The term Agent OS has spent a year gaining traction across research and product circles. It describes a layer that goes well beyond an agent library: request scheduling, context management, persistent memory, and isolation. A look at the real state of that concept.
Model Context Protocol turns ten months old since Anthropic's announcement, and it is no longer just a proposal: hundreds of servers, cross-vendor implementations and a public registry now back it. A look at what has worked, what is still weak, and why 2025 marks the shift from curiosity to basic infrastructure.
After months of rumors, OpenAI released GPT-5 in early August. The first weeks of real-world use show a picture less spectacular than the marketing suggested and more useful than many expected. It is worth separating what is genuinely new from what is merely incremental.
Small language models have become genuinely useful. Phi-3.5, Gemma 2, and Llama 3.2 fit on modest hardware and solve bounded tasks without reaching the cloud. A look at where they fit on the factory floor and when skipping the large model pays off.
RAG 2.0 means retrieval built from several sources at once rather than a single vector search: dense embeddings, lexical matching, and knowledge graphs that capture relationships between entities, with a reranking layer ordering the final candidates. The 2023 pattern of one vector database plus an LLM no longer describes what production systems actually do.
Gemini 2.5 Pro reached preview on 25 March 2025 and general availability at the end of June, alongside the cheaper, faster Gemini 2.5 Flash. Two things separate it from Gemini 2.0: a one-million-token context window that behaves stably to at least 500k, and multimodality that has left the demo stage behind.
Anthropic released Claude Opus 4 and Claude Sonnet 4 on 22 May 2025, the first major naming jump since the 3.5 series. Claude 4 reasons noticeably better over long programming tasks: multi-hour refactors that previously stalled without a human nudge now run further alone, and the family targets agentic, multi-step flows.
For a decade, knowledge graphs were an academic idea with few real use cases, held back by the cost of building and maintaining the schema. LLMs have changed that equation: they now extract entities automatically and help anchor answers, audit reasoning, and support agents without hallucinating.
Continuous RAG evaluation catches the quiet kind of failure: the system never goes down, never returns errors, never trips a latency alert, it simply answers worse as the index, the model, and user questions drift. Track retrieval precision, effective recall, faithfulness to the retrieved context, answer relevance, and p99 latency.
AI agents have moved from a lab curiosity to serious SDKs from three major providers. A reflection on moving from the flashy demo to an internal use case that shifts a real, measurable metric.
Graph RAG layers an explicit graph of entities and relationships on top of retrieval, so a question can be answered by traversing connections rather than by vector similarity alone. Microsoft Research published the GraphRAG paper in April 2024 and open-sourced the code that July. Lighter variants such as LightRAG and HippoRAG followed, running on Neo4j or Memgraph.
Claude 3.7 Sonnet, released by Anthropic on February 24, is a careful refinement rather than a generational jump. The same model answers in standard mode or in an extended thinking mode you switch on per request, trading tokens and latency for better results on hard problems. It also ships Claude Code, a command-line tool for programmers.
vLLM remains the reference engine for serving LLMs on GPU in 2025: automatic prefix caching sharply cuts latency for repeated prompts, speculative decoding speeds up large models, and multi-LoRA support lowers the cost of multi-tenant SaaS, though multi-GPU support and non-NVIDIA hardware remain weak points.
GraphRAG has been in real enterprise use for over a year: during indexing, an LLM builds a knowledge graph that answers global questions about a corpus well, precisely where classic RAG fails because no single chunk holds the full answer. Here I compare indexing costs, the cases where it pays off, and the hybrid pattern that teams have settled on.
Three years after RLHF became popular, the model-alignment field is far richer. A review of RLHF, DPO, and newer methods such as KTO and ORPO, with criteria for choosing between them.
Google released Gemma 2 in mid-2024, and it has since seen real production use. A look at how it competes in the open-model ecosystem, which sizes actually make sense, and where its adoption has settled in.
o3-mini, the first public release of OpenAI's o3 reasoning series, clearly improves logic, math, and complex code over GPT-4o, though it answers slower and still hallucinates facts. This analysis, based on weeks of real use, explains where it pays off and where it does not.
Gemini 2.0, announced by Google in December, puts tool use and agent behavior at the center of the product: it is designed to execute actions, not only generate text. Its clearest advantages are Flash's one-million-token context window, cheap input tokens, and first-class access to Search, Maps, Cloud, and Workspace. Claude 3.5 Sonnet still leads on complex reasoning.
Qualcomm, Intel and AMD Copilot+ processors have normalised the presence of an NPU in everyday PCs. A 40 TOPS NPU can run quantised Phi-3 Mini drawing just 5-10 W, versus 40-50 W for a laptop GPU doing the same task. What actually changes for running AI models locally, and when it is worth it.
LoRA cuts fine-tuning cost for large language models by training only small low-rank adaptation matrices instead of every parameter in the base model. QLoRA adds 4-bit quantization on top, cutting required GPU memory by 65-75%, with quality loss of just 1-3% versus full fine-tuning.
Claude 3.5 Sonnet (Anthropic, June 2024) matches Claude 3 Opus quality at Sonnet pricing, with a 200k-token context window and 92% on HumanEval. It stands out in coding and complex instruction-following, and introduced the Artifacts workspace feature on Claude.ai.
Mistral Large 2, released by French startup Mistral AI in July 2024, is a 123-billion-parameter model with a 128k-token context window that rivals GPT-4o and Claude 3.5 Sonnet on several benchmarks. Its EU data residency and its 3 EUR per million input tokens pricing make it the most serious European alternative to US providers.
GPT-4 Turbo, released in November 2023, expanded GPT-4's context to 128,000 tokens and cut the input price threefold, down to 10 dollars per million tokens. GPT-4o now beats it on price, speed and answer quality, but Turbo still holds up in stable production apps, contracts pinned to a specific version, and deterministic tests that depend on its exact behaviour.
Outlines, Guidance e Instructor obligan al modelo a emitir JSON válido en el propio paso de generación. Cuándo ganan frente a reintentos y function calling.
Mixtral 8x22B is Mistral AI's Mixture of Experts model released in April 2024: 141B total parameters but only 39B active per token, an unrestricted Apache 2.0 licence, and multilingual performance ahead of Llama 3 70B in Spanish, French, Italian, and German. Production serving needs datacenter-class GPUs.
Claude 2, launched by Anthropic in July 2023, offers a 100,000-token context window and safety grounded in Constitutional AI. Against GPT-4 it wins on long-document analysis and wide-context code; GPT-4 remains ahead on complex mathematical reasoning and its tooling ecosystem.
LangChain is a Python framework that unifies building LLM applications: prompt templates, retrievers over vector databases, function-calling agents, and conversational memory. It earns its keep in fast prototypes and multi-model systems, but for a single well-defined production use case, direct code usually stays more maintainable.
Function calling converts the LLM into a component that returns structured data. How it works, real use cases, and common production pitfalls.
Five months after launch, GPT-4 excels at chained reasoning, technical writing, and medium-complexity code, but still fails at arithmetic, post-cutoff information, and cross-conversation consistency. Claude 2 wins on long context; LLaMA 2 wins on cost and privacy.
Google launched Bard in February 2023 with PaLM 2 as its answer to ChatGPT, unveiling the model in May the same year in four sizes: Gecko, Otter, Bison, and Unicorn. PaLM 2 competes with GPT-3.5 and GPT-4 on benchmarks like MMLU and BIG-bench, but Google's real edge is Workspace integration, not the model itself.
Fine-tuning your own LLM pays off in three cases: you need a very specific style or voice, a rigid structured output format, or you want lower cost and latency from a small specialised model. LoRA and QLoRA have cut the GPU cost, but preparing data and running the model in production are still expensive. For everything else, RAG and prompt engineering are usually enough.
