Category
AI without the hype: models, agents and use cases that work in production.
Related paths
For a decade, knowledge graphs were an academic idea with few real use cases, held back by the cost of building and maintaining the schema. LLMs have changed that equation: they now extract entities automatically and help anchor answers, audit reasoning, and support agents without hallucinating.
In AI systems the real cost is not EC2 instances but input tokens in RAG and agents, chained tool calls, and frequent reindexing; those vectors, plus unattributed experimental spend, concentrate most of the monthly production bill.
Continuous RAG evaluation catches the quiet kind of failure: the system never goes down, never returns errors, never trips a latency alert, it simply answers worse as the index, the model, and user questions drift. Track retrieval precision, effective recall, faithfulness to the retrieved context, answer relevance, and p99 latency.
Crunchbase and CB Insights first-quarter data confirm that global startup funding has rebounded, but nearly all of the growth is concentrated in startups presenting themselves as AI. The rest of the ecosystem remains in correction.
The LLM wrappers that survived the 2022 to 2024 startup wave own something their underlying model cannot supply: proprietary data, network effects, or a workflow users already live inside. Everything else was a thin prompt layer over an API, and structurally bad margins killed it as inference costs rose with usage.
LLM agent security became a real incident category the moment assistants gained tool access. Agents that call tools expose a far wider attack surface than chatbots, and it now carries assigned CVEs, published audit reports, and its own OWASP Top 10. The spread of the Model Context Protocol and agent-driven corporate workflows built that surface in under 18 months.
AI agents have moved from a lab curiosity to serious SDKs from three major providers. A reflection on moving from the flashy demo to an internal use case that shifts a real, measurable metric.
Graph RAG layers an explicit graph of entities and relationships on top of retrieval, so a question can be answered by traversing connections rather than by vector similarity alone. Microsoft Research published the GraphRAG paper in April 2024 and open-sourced the code that July. Lighter variants such as LightRAG and HippoRAG followed, running on Neo4j or Memgraph.
The AI features Figma has rolled out since Config 2024 are changing how product design teams work. A look at what each feature delivers, what remains human work, and which habits are taking hold across teams.
AI governance in a company means a standing committee, written policies, a model and use-case inventory, risk assessment, and audits. The first provisions of the EU AI Act took effect on 2 February 2025, banning practices such as social scoring and requiring minimum AI literacy for staff. Fines reach 35 million euros or 7 percent of global turnover.
Claude 3.7 Sonnet, released by Anthropic on February 24, is a careful refinement rather than a generational jump. The same model answers in standard mode or in an extended thinking mode you switch on per request, trading tokens and latency for better results on hard problems. It also ships Claude Code, a command-line tool for programmers.
A year ago open weights were a gamble; today they are a real production option. I review what has worked, what has not, and how Llama, DeepSeek, Qwen, and Mistral are fitting into enterprise architectures that used to depend on closed APIs.
vLLM remains the reference engine for serving LLMs on GPU in 2025: automatic prefix caching sharply cuts latency for repeated prompts, speculative decoding speeds up large models, and multi-LoRA support lowers the cost of multi-tenant SaaS, though multi-GPU support and non-NVIDIA hardware remain weak points.
GraphRAG has been in real enterprise use for over a year: during indexing, an LLM builds a knowledge graph that answers global questions about a corpus well, precisely where classic RAG fails because no single chunk holds the full answer. Here I compare indexing costs, the cases where it pays off, and the hybrid pattern that teams have settled on.
Three years after RLHF became popular, the model-alignment field is far richer. A review of RLHF, DPO, and newer methods such as KTO and ORPO, with criteria for choosing between them.
Google released Gemma 2 in mid-2024, and it has since seen real production use. A look at how it competes in the open-model ecosystem, which sizes actually make sense, and where its adoption has settled in.
o3-mini, the first public release of OpenAI's o3 reasoning series, clearly improves logic, math, and complex code over GPT-4o, though it answers slower and still hallucinates facts. This analysis, based on weeks of real use, explains where it pays off and where it does not.
Gemini 2.0, announced by Google in December, puts tool use and agent behavior at the center of the product: it is designed to execute actions, not only generate text. Its clearest advantages are Flash's one-million-token context window, cheap input tokens, and first-class access to Search, Maps, Cloud, and Workspace. Claude 3.5 Sonnet still leads on complex reasoning.
Two years running AI-assisted code review in a real team leave a clear balance: AI catches mechanical oversights well and writes useful pull-request summaries, but it struggles with architectural judgment and produces many false positives on subtle bugs. The single decision that helped the most was not blocking merges on its automated comments.
Meta released Llama 3.2 with models as small as 1B and 3B parameters, built specifically to run on the device rather than in a datacentre. A look at what they can genuinely do, where the small sizes fall short, and how they compare with the alternatives.
Qualcomm, Intel and AMD Copilot+ processors have normalised the presence of an NPU in everyday PCs. A 40 TOPS NPU can run quantised Phi-3 Mini drawing just 5-10 W, versus 40-50 W for a laptop GPU doing the same task. What actually changes for running AI models locally, and when it is worth it.
Measuring RAG quality rigorously takes more than skimming a handful of answers: it requires objective metrics (faithfulness, relevance, context precision, and coverage), a golden set of hundreds of curated questions, and regular human validation of the LLM judge to avoid misleading conclusions.
OpenAI introduced o1 in September 2024: a model that reasons internally before answering. What changes, and when the extra cost is worth it.
Hybrid search combines BM25 and vector retrieval to cover what each misses alone. Vectors fail on exact identifiers like SKUs or CVEs; BM25 fails when query and document use different vocabulary for the same idea. Reciprocal Rank Fusion (RRF) merges both rankings without depending on their score scales.
llama.cpp is the C++ library that powers Ollama and much of the local-LLM ecosystem. 2024 added speculative decoding with two- to three-fold speedups, an RPC server for sharding layers across machines, and a stable GGUF format. Ollama covers 90% of cases; going direct pays off with uncommon hardware or specific flags.
Ollama became the standard for running large language models locally in 2024. It wraps llama.cpp in a single binary with Docker-style CLI and an OpenAI-compatible API. Phi-3 Mini runs in 4 GB; Llama 3.1 8B Q4 needs 6 GB. For production traffic at scale, vLLM remains the correct choice.
Model Context Protocol (MCP) is the open standard Anthropic published on 25 November 2024 to connect language models with external data and tools over JSON-RPC 2.0. It does not replace function calling: it standardises the server side, aiming to become for context what the Language Server Protocol is for code editors.
Product-market fit for LLM-powered products still depends on the same classic signals: cohort retention, NPS, and revenue expansion. What changes are the higher quality baseline, faster competitor iteration, and where durable moats come from: proprietary data, workflow integration, and network effects.
LLM applications need three distinct observability planes: prompt and response traces for debugging hallucinations, per-token and per-feature cost tracking, and response quality evaluation. Mature tools like Langfuse, LangSmith, and Helicone cover all three planes with specific instrumentation.
TensorRT-LLM is the NVIDIA inference engine that compiles each model into a binary optimized for the exact GPU and batch size it will serve. It uses hand-written CUDA kernels and native FP8 quantization on H100. Against vLLM it can run 2 to 3 times faster in the best case, at the cost of a 30 to 90 minute build.
LoRA cuts fine-tuning cost for large language models by training only small low-rank adaptation matrices instead of every parameter in the base model. QLoRA adds 4-bit quantization on top, cutting required GPU memory by 65-75%, with quality loss of just 1-3% versus full fine-tuning.
Computer Use is the Claude API feature, launched by Anthropic on 22 October 2024, that lets the model view screenshots and move the mouse, type, and click inside a loop your own system executes and controls. It works well on apps without an API and fails on CAPTCHAs, highly dynamic interfaces, and long tasks.
GitHub Copilot Workspace, in technical preview since April 2024, proposes task-oriented development: describe the problem in a GitHub issue and the AI reads the codebase, generates an editable multi-file plan, and implements it. It competes with Cursor Composer, though with more latency; its edge is native integration with PRs, issues, and GitHub history.
OpenAI published Swarm as an experimental, educational framework for multi-agent systems. It reduces coordination to two concepts — agents and handoffs — and fits in under 500 lines of Python. A comparison with CrewAI and LangGraph.
vLLM serves language models on GPU using PagedAttention and continuous batching, two techniques that multiply throughput compared with a naive server. It exposes an OpenAI-compatible API, so migrating an existing application only requires changing the base URL and deploying the right binary.
Claude 3.5 Sonnet (Anthropic, June 2024) matches Claude 3 Opus quality at Sonnet pricing, with a 200k-token context window and 92% on HumanEval. It stands out in coding and complex instruction-following, and introduced the Artifacts workspace feature on Claude.ai.
Mistral Large 2, released by French startup Mistral AI in July 2024, is a 123-billion-parameter model with a 128k-token context window that rivals GPT-4o and Claude 3.5 Sonnet on several benchmarks. Its EU data residency and its 3 EUR per million input tokens pricing make it the most serious European alternative to US providers.
After two years of RAG in production, clear patterns emerge: smart chunking, hybrid search, re-ranking, continuous evaluation. What to avoid.
CrewAI models agents as a team, each with an explicit role and a set of tasks to carry out. How it compares with LangGraph and AutoGen, what the multi-agent pattern really costs to run, and when the extra coordination is worth paying for.
OpenAI's Assistants API offers persistent threads, sandboxed code execution, and managed document search, but OpenAI is shutting it down completely on August 26, 2026 in favor of the Responses API. We look at when it used to pay off against Chat Completions with your own infrastructure, and what to do if your project still depends on it.
The EU AI Act (Regulation 2024/1689) entered force on 1 August 2024. It classifies AI systems into four risk levels with graduated deadlines: prohibitions in February 2025, GPAI obligations in August 2025, and high-risk requirements in August 2026. It applies to any company operating or selling in the EU, with fines exceeding GDPR levels.
Installing Ollama on an Apple Silicon Mac is as simple as running one Homebrew command. Then pick a model based on available RAM (Phi-3 for 8 GB, Llama 3.1 8B for 16 GB) and expose the local, OpenAI-compatible HTTP API on port 11434 to plug it into your own applications.
GitLab Duo brings native AI into the whole devops flow: code completion, chat, MR summaries and vulnerability explanation. Duo Pro costs 19 dollars per user monthly on top of Premium or Ultimate, the same as GitHub Copilot Business. It pays off when your team already lives in GitLab.
Meta released Llama 3.1 405B on July 23, 2024: 405 billion parameters, 128k token context, and benchmarks matching GPT-4o and Claude 3.5 Sonnet. Self-hosting needs about 220 GB of VRAM in Q4; Together.ai, Fireworks, and Groq offer per-token access.
Embeddings retrieve fast but rank poorly. A reranker over the top-100 lifts precision 15–30 %. When it pays off and when it does not.
GPT-4 Turbo, released in November 2023, expanded GPT-4's context to 128,000 tokens and cut the input price threefold, down to 10 dollars per million tokens. GPT-4o now beats it on price, speed and answer quality, but Turbo still holds up in stable production apps, contracts pinned to a specific version, and deterministic tests that depend on its exact behaviour.
Evaluating a RAG system without metrics is pure guesswork. Ragas measures four core signals: faithfulness, answer relevancy, context precision and context recall, using an LLM as judge. TruLens, DeepEval and other frameworks cover similar ground. Wiring evaluation into CI from day one catches regressions in prompts, chunking or model choice before they reach production.
SGLang adds a Python DSL for controlling LLM generation with constrained decoding, parallel branching, and RadixAttention, the structure that indexes the KV cache as a radix trie to reuse shared prefixes across requests. When that pattern exists, speedups over vLLM reach up to 5 times; without it, the advantage shrinks.
