Categories
Intermediate

Local LLMs: run models on your own hardware

Run language models on your own machine — no API keys, no quotas and your data never leaves home. The path goes from installing Ollama to fine-tuning on an Apple Silicon Mac, and ends by giving your model memory with a vector database for RAG.

  • 5 resources
  • ~33 min read

  1. How to Install Ollama to Run LLMs on Your Computer

    Ollama makes it trivial to run models like Llama 2 or Mistral on your own computer: one binary, one command, and quantised weights downloading to disk with no compilation required. Covers installation on macOS, Linux, and Windows with an honest look at what local inference can and cannot do compared to frontier models.

    • 5 min

  2. How to Install Ollama on macOS with Apple Silicon

    Installing Ollama on an Apple Silicon Mac is as simple as running one Homebrew command. Then pick a model based on available RAM (Phi-3 for 8 GB, Llama 3.1 8B for 16 GB) and expose the local, OpenAI-compatible HTTP API on port 11434 to plug it into your own applications.

    • 7 min

  3. How to install and tune oMLX on M5 Max 128 GB

    Tested May 2026 recipe: oMLX 0.3.8 on Mac M5 Max with 128 GB, TurboQuant at 3.5-bit, Qwen 3.6 35B-A3B model stack, Claude Code wiring and real benchmarks.

    • 12 min

  4. How to Install PostgreSQL with pgvector Step by Step

    This guide installs PostgreSQL 16 with pgvector on Debian or Ubuntu using the official PGDG repository, creates a dedicated role and database, tunes memory for production, and explains when the HNSW index beats IVFFlat depending on vector volume and the available maintenance window.

    • 8 min

  5. Local LLM Calculator

    Will that model fit your GPU? Is self-hosting cheaper than the API? Estimate the VRAM and the cost break-even point.

    • 1 min