Tag
DeepSeek Harness (dsh) is the open-source agent harness DeepSeek released in August 2026. It works with a local model if you declare an openai-completions provider in settings.yaml that points at llama-server, with a placeholder key and the real context size. With Qwen3.5-4B on CPU it left the tests green in 4 of 5 attempts, but slowly.
Colibri is an Apache 2.0 inference engine written in C that runs Mixture-of-Experts models of up to 2.8 trillion parameters without loading them whole: it keeps the dense part in RAM and reads from the SSD only the experts the router picks for each token. That is why its speed depends on the disk, the cache and free RAM.
FreeToken is an open-source inference engine, released by UC Berkeley and UT Austin researchers in August 2026, that splits a MoE model's experts across GPU, CPU and system memory according to each machine's measured bandwidth. It serves models from 35B to 753B parameters on consumer hardware.
NPUs stopped being an accessory and became the component that defines real performance in laptops, phones, and small servers. A practical look at the hardware that rules 2026, which workloads pay off, and where the traditional GPU still wins.
Phi-3 is Microsoft Research's family of small language models aimed at the edge, and it competes directly with Llama 3.2, Gemma 2 and Qwen 2.5. Phi-3-mini holds 3.8B parameters and, quantized to 4 bits, fits in about 2 GB, running on a CPU with a neural accelerator, an integrated GPU or an NPU.
Neural processing units have stopped being a marketing label on Snapdragon, Apple Silicon and AMD Ryzen AI laptops. Here is what you can actually do from code today, which tools are mature, and when it pays off to target the NPU instead of the CPU or GPU.
