What is oMLX and how it differs from MLX, Ollama and LM Studio

oMLX is a local inference server for Apple Silicon Macs that turns Apple’s MLX framework into a permanent service speaking the APIs your tools already use. The commonest confusion is mistaking it for Apple’s framework, from which it differs by a single letter. This article clears up what each one is, how it compares with Ollama and LM Studio, and why there is no Windows or Linux build.

Key takeaways

• oMLX is a server, not a framework. It wraps MLX, Apple’s compute library, in a FastAPI process with web endpoints.
• It is a native macOS app written in Swift and SwiftUI, driven from the menu bar, under the Apache 2.0 licence.
• Its technical differentiator is the tiered KV cache: hot blocks in RAM and cold blocks on SSD that survive restarts.
• It keeps several models loaded at once and evicts the least recently used one when memory runs short, under a default ceiling of system RAM minus 8 GB.
• It only runs on Apple Silicon with macOS 15 or later. There is no Windows, Linux or Docker build, and there will not be, because of how MLX is built.

What oMLX actually is

oMLX is a language model inference server that runs on your Mac and exposes those models over HTTP. The project’s own description puts it as an inference server with continuous batching and SSD caching for Apple Silicon, managed from the macOS menu bar.

Underneath it is a FastAPI process wrapping MLX, the machine learning framework Apple published for its M-series chips. On top of that base it adds three things MLX does not carry: an engine pool that keeps several models in memory and evicts the least used when room is needed, a key-value cache split between RAM and SSD, and a set of endpoints that imitate the OpenAI and Anthropic APIs.

It ships under the Apache 2.0 licence in the jundot/omlx repository. The current version is 0.6.4, from 29 August 2026, and the release cadence is high: six versions shipped between 18 and 29 August.

The app is native, written in Swift and SwiftUI, not a browser wrapper. It lives in the menu bar, starts and stops the server, keeps serving stats across restarts and updates itself.

oMLX is not MLX

This is the distinction that drives the most searches and it deserves a direct answer.

MLX is Apple’s framework: an array computation library designed for the unified memory of M-series chips, with an API resembling NumPy and PyTorch. It is the layer that runs the mathematical operations. On its own it serves no models and knows nothing about HTTP.

mlx-lm is the layer above: it loads language models, runs them and generates text. It is what you use if you are writing a Python script.

oMLX is the service layer: it takes mlx-lm, puts it in a process listening on a port, manages memory across several models and speaks the protocols your clients already know. It is the difference between having a library and having a service.

Put another way: MLX runs the model, oMLX decides which model is loaded, how much memory it takes, who may call it and in what format.

How it works underneath

Three pieces explain almost all of its behaviour.

The engine pool manages the lifecycle of language models, vision models, embedding models and rerankers. It applies least-recently-used eviction, lets you pin models so they are never unloaded, and supports a per-model time to live that frees one after an idle period. A memory enforcer imposes the total ceiling, which defaults to system RAM minus 8 GB, so the whole Mac does not run out of memory.

The tiered key-value cache is its distinguishing feature. It works in blocks, with prefix sharing, copy-on-write and least-recently-used eviction, a design borrowed from vLLM. Hot blocks live in RAM; when that fills, cold ones spill to SSD in safetensors format. If you send another request with the same prefix, the blocks are restored from disk rather than recomputed, and that keeps working after a server restart. On top of that, TurboQuant quantises the cache with a mean-squared-error codec and cuts its size by 60% to 75%.

Continuous batching lets it serve concurrent requests without waiting for the previous one to finish, through mlx-lm’s batch generator. The limit is set at eight concurrent requests and can be changed.

Compared with Ollama, LM Studio and llama.cpp

The four tools solve the same problem with different priorities:

The practical reading is simple. If you work across several machines or on Linux, Ollama is the obvious pick and I have written about installing it on a Mac. If you want a desktop app to explore models in, LM Studio fills that gap. If you care about squeezing performance by hand, llama.cpp optimisations go further.

oMLX wins when the machine is a Mac with plenty of memory and you want to treat it as a server: several models alive, concurrent requests, a cache that survives restarts and clients speaking both the OpenAI and the Anthropic dialect. That last point is what makes it attractive to anyone using Claude Code, because the client points at the Mac with no translation in between.

Why there is no Windows, Linux or Docker build

The short answer is that oMLX is MLX, and MLX is Apple’s.

MLX is built on the unified memory and the Metal engine of M-series chips. It is not a compatibility layer you can recompile: the memory model that makes it fast, where the processor and the graphics unit share the same space with no copies, does not exist on a PC with a dedicated graphics card.

That is where the requirements come from, which the Homebrew formula checks and enforces: macOS 15 or later, Python between 3.11 and 3.13, and an Apple Silicon chip from the M1 to the M5. An Intel Mac is rejected during installation.

There is no Docker image either, for the same reason: a container on macOS runs inside a Linux virtual machine with no access to Metal. Putting oMLX in a container would strip it of the acceleration that justifies its existence.

If your hardware is not a Mac with an M-series chip, the question is not how to install oMLX but what to use instead, and the answer is usually Ollama or llama.cpp.

Who it makes sense for

It makes sense if you have an Apple Silicon Mac with plenty of unified memory and want it to act as a model server for several tools at once. The case that justifies it best is somebody using a coding assistant who wants it talking to a local model without giving up the client they already know.

It does not make sense if you only want to try a model now and then, because Ollama or LM Studio give you that with fewer moving parts. Nor if you need the same setup on Linux, nor if your Mac has 16 GB, where the memory ceiling leaves little room once the 8 GB reserve is subtracted.

And it is worth keeping in mind: the project is young and moves fast, with several releases a month touching the inference engine. That is good for performance and less good for stability. Treat it as a working tool you upgrade after reading the notes, not as something you install and forget.

Frequently asked questions

Are oMLX and MLX the same thing?

No. MLX is Apple’s compute framework for its M-series chips, the layer that runs the operations. oMLX is a separate server, from the developer jundot, that wraps MLX and exposes it over HTTP through the OpenAI and Anthropic APIs, adding memory management, on-disk caching and several models loaded at once.

Can I install oMLX on Windows or Linux?

No, and it is not planned. oMLX depends on Apple’s MLX framework, which relies on the unified memory and Metal of M-series chips. The requirements are macOS 15 or later, Python between 3.11 and 3.13 and an Apple Silicon chip. On other platforms the equivalent alternative is Ollama or llama.cpp.

Which models does oMLX support?

Any language model from the mlx-lm catalogue; vision models such as the Qwen3.5 series, GLM-4V and Pixtral; text-recognition models such as DeepSeek-OCR, DOTS-OCR and GLM-OCR, which it auto-detects; embedding models such as BERT, BGE-M3 and ModernBERT; and rerankers based on ModernBERT and XLM-RoBERTa.

Conclusion

oMLX occupies a fairly narrow gap and occupies it well: turning a Mac with memory to spare into a serious model server, with several loaded at once, a cache that persists and compatibility with the two API formats that dominate the market. That narrowness is deliberate, and it explains both what it does well and why it will never leave macOS.

If it fits, the next step is installing it: the Homebrew route is in installing, updating and uninstalling oMLX, and server configuration in the API and port guide. The Spanish version of this article is at Qué es oMLX y en qué se diferencia de MLX, Ollama y LM Studio.

Sources

1. jundot/omlx, official repository and documentation
2. oMLX 0.6.4 release notes
3. MLX, Apple’s machine learning framework
4. mlx-lm, the language model layer on top of MLX
5. vLLM and PagedAttention, origin of the block-based cache
6. DeepWiki: oMLX internal architecture