Which open models of September 2026 are worth running on your own hardware

September 2026 was a thin month for anyone who runs models on their own machine, and that in itself is news. Neither Qwen, Mistral, Google nor Meta published a new text model in their Hugging Face organisations. What did arrive at home-machine size I downloaded and ran on an arm64 box with no GPU, using llama.cpp v0.5.0 and five fixed tasks in Spanish. This review says what shipped, under which licence, and what each model did when I gave it real work.

Key takeaways

• Xiaomi’s MiMo-V2.6-Distill-Qwen-9B is the release of the month worth disk space: 9.41 billion parameters, vision, and an MIT licence declared on the model card.
• In my tests MiMo chained both tool calls correctly, returned clean JSON and read a docker ps screenshot, but it invented the --auto-heal option in all five attempts with the default sampling.
• MiniCPM5-2B, from 6 September, weighs 1.56 GB in Q4_K_M, chained the same tools as MiMo and rejected the made-up option in four attempts out of five.
• DFM Mimir v1.5 is a Danish model with 1.787 billion parameters: in Spanish it answers with empty sentences and invents Docker commands.
• The month’s large models (DeepSeek-V4.1-Flash, MiMo-V2.6-Pro-RL, Intern-S2) have open weights, but between 403 billion and more than a trillion parameters.

What this review covers and how I tested it

It covers what was published between 1 and 28 September 2026, and the data was checked on 28 September. For each model I read its Hugging Face card, the repository’s public API (creation date, licence, parameter count) and the LICENSE file when there is one. It is the same method as the review of the open models of August 2026, with one difference: this time I ran the small models.

The test is deliberately modest. An aarch64 machine with 18 cores and no GPU, shared with other jobs, the llama.cpp server from its official server-v0.5.0 image, and Q4_K_M quantisations. Five identical tasks for every model, at temperature 0:

1. Explain in five sentences what a reverse proxy is and when to pick Traefik over Nginx
2. Restart a container and check free memory using two declared tools
3. Extract a JSON object with three keys from a docker log with exit code 137
4. Write a Python function to_bytes() that I grade with seven cases
5. Explain what docker run --auto-heal does, an option that does not exist

Five prompts are not a benchmark. They show how a model behaves when you hand it real work, which is what the tables on model cards do not show. I measured speed and memory separately, on an idle machine, and they have their own section below.

What shipped with downloadable weights

These are the September models you can download today. The size column is the Q4_K_M GGUF file where one exists:

The first four rows are the ones that matter if you have a laptop or a home server in front of you. The rest are open weights with clean licences that need a data-centre GPU server.

The big labs spent September in silence

The most striking fact of the month is an absence. I listed their Hugging Face organisations by creation date, and this is what is there:

• Qwen: image models only, the Qwen-Image-2.1 family
• Mistral: nothing new since 16 July
• Google: only GNM v3, a parametric 3D model of human heads, on 1 September
• IBM: only MLX conversions of Granite 4.2
• Meta: still no Muse Spark 1.2 repository, which it announced as open in August

I owe the previous review a correction: DeepSeek-V4-Flash-Vision-Exp appeared in August as an API-only model, because on 30 August it had no repository. Its weights went up on 31 August, MIT licensed with 304,647 million parameters, and ggml-org uploaded GGUF files on 2 September. The rule still holds (no repository in the official organisation means no weights), except that a repository can appear the next day.

How the models handled five real tasks

MiMo and MiniCPM5 tied on tool calls, MiMo was the only one to return correctly typed JSON, and none of the three is reliable at writing code or at a trick question. This is the first pass, at temperature 0:

That first pass is misleading, because at temperature 0 there is no single answer. llama.cpp batches requests that arrive together, and that changes the rounding: I repeated the code task with MiMo and the second version passed 6 of 7 cases. So I repeated the trap five times, and the function and the JSON three times, with the sampling each GGUF ships with, which is the one you will use:

The trap is the result that flips. MiMo rejected it at temperature 0 and invented it all five times with the default sampling, while MiniCPM5 did exactly the opposite.

MiMo-V2.6-Distill-Qwen-9B, the release worth having

Xiaomi presents it as a fine-tune of Qwen3.5-9B on data generated by its large models, and says so plainly: "We release this SFT checkpoint as a starting point for open research in agentic reinforcement learning". It is a starting point for research, not a polished product, and that shows both ways. According to its card it goes from 32.0 to 44.6 on SWE Pro and from 25.9 to 35.2 on Toolathlon-Verified compared with the original Qwen3.5-9B.

In my tasks the agentic part worked. It requested restart_container with the argument jellyfin and, once I returned the result, called host_memory without a reminder. The JSON came back with the exit code as an integer and the cause as a Spanish sentence. It also read a docker ps screenshot: it spotted the container stuck in a restart loop and the one with a failing health check, although it read (137) as "137 restarts" when it is the exit code.

The screenshot is from the llama.cpp web UI, cropped to remove the speed figures. In that version of the answer it no longer misreads the exit code, but it ends by recommending docker healthcheck, which is not a Docker command.

What fails is the edge of its knowledge. On the first pass it rejected --auto-heal and recommended --restart unless-stopped, but in the same answer it suggested --healthcheck-interval and --auto-rm, which do not exist either (the real ones are --health-interval and --rm). With the default sampling it invented the option in all five attempts. Three times it moved it to docker service create, which does not have it either, and twice it called it an alias of --restart=always.

MiniCPM5-2B as the small reference

MiniCPM5-2B is from this month and I already measured it in depth in the article on CPU-only LLMs on arm64, so here it serves as the yardstick. With 2.517 billion parameters, its first to_bytes() passed all seven cases, including 2.5 GiB with a dot and 0,5 MB with a comma, although only one of the three repeats came close (6 of 7). On tools it did the same as MiMo: the restart first and, with the result in hand, the memory.

Where it falls short is anything that needs judgement. The Spanish explanation was correct but hollow ("Traefik offers a smoother experience"). The JSON returned the exit code as a string and the cause as the whole kernel message copied verbatim.

At temperature 0 it answered the trap with confidence: --auto-heal restarts the container when the health check fails and is enabled with true. Its own reasoning said "I’m not 100% sure" right before asserting it. With the default sampling it rejected the option in four attempts out of five, and in one of them it offered a --update flag instead, which docker run does not have (the real option is --pull).

DFM Mimir v1.5, outside its language

Mimir is a serious project from Danish Foundation Models, trained from scratch on Danish and English. Its recurrent architecture (HRM-Text) reuses its 16 layers in eight passes (six of the L module and two of the H module) per complete reasoning pass. llama.cpp supports it since v0.5.0, released on 23 September. I tested it in Spanish because that is what matters here, and also in English and Danish to be fair.

In Spanish it is no use: it repeated the same idea about Traefik and Nginx three times, wrote codice_salida instead of codigo_salida, and its Python function does not even run. In English and Danish the answers were cleaner but just as generic, and in English it also explained --auto-heal as if it existed. Tool tasks fail too: it reasoned out loud, invented a docker memory command and emitted no call. When I gave it the result of the finished restart, it replied that it had no access to host memory instead of calling host_memory.

How fast they run on an arm64 CPU without a GPU

MiniCPM5-2B generated a median 56.85 tokens/s and MiMo-V2.6-Distill 18.03: the model that gets more right costs three times as much per token. I measured the four models on 28 September 2026, one after another with the machine to myself, using the llama.cpp v0.5.0 server with Q4_K_M and 6 threads. Each figure is the median of 5 requests after one warm-up, with a prompt of 761 to 841 tokens depending on the tokenizer, 256 generated tokens and one request at a time:

The one-minute load average was below 3 (between 2.45 and 2.96) when each model started; during the run it rose to 9.8 from llama.cpp itself. Memory is the docker stats reading at the end, with the default 8,192-token context (4,096 for Mimir) and MiMo without the vision projector. I used 6 threads because on this machine they beat 16, as I measured in the article on CPU-only LLMs on arm64. There MiniCPM5 reached 83.19 tokens/s with another llama.cpp version and another prompt, so only compare figures within one table.

Two results stand out. Nex-N2.5-mini, with 35,107 million parameters, generates faster than MiMo because it activates only 8 of its 256 experts per token. Mimir, with less than half of MiniCPM5’s parameters, runs more than four times slower, because its recurrent architecture runs its 16 layers eight times per complete reasoning pass.

The models that are not what they seem

Five September releases appear in "new models" lists and are not an assistant you can chat with:

• Intern-Decision (InternLM): scores the options of a schema in a single pass. Its card says it: "It does not call generate() or sample free-form text"
• LFM2.5-VL-3B-DSpark (Liquid AI): a 279.5-million-parameter drafter for speculative decoding of LFM2.5-VL-3B, not a model you use on its own
• EVIE-4.5B and WeVisDoc (Tencent): visual document retrieval, for search over scanned PDFs
• LensVLM-9B (Apple): same base and size as MiMo, but its licence restricts it to research
• VibeVoice-ASR-Streaming (Microsoft): speech recognition with speaker attribution in 10 languages, Spanish included, at 1.5 and 7 billion parameters

Nex-N2.5-mini deserves a separate mention. It is a mixture-of-experts (MoE) model with the Qwen3.5-35B-A3B architecture, 256 experts and 8 active per token, which its maker deploys on two H100s.

As a GGUF it comes down to 22.32 GB in Q4_K_M, so it fits on a machine with 32 GB of memory. On my machine it generated 25.15 tokens/s. I only gave it three of the five tasks: it chained both tools and returned correct JSON, but faced with the made-up option it used up its 3,000 tokens reasoning without answering.

What each licence actually permits

The licence shown on the Hugging Face page does not always have a file behind it, and that matters if you are putting the model in a product. This is what I found when opening each repository:

• MIT without a file: MiMo-V2.6-Distill-Qwen-9B declares license: mit in the card metadata, but the repository has no LICENSE file. Nex-N2.5-mini does the same with Apache 2.0. It is a valid statement by the author, though for commercial use keep a copy of the card as it stood
• Apache 2.0 with a file: DFM Mimir and Intern-Decision include the full text
• MIT with a file: DeepSeek-V4.1-Flash
• Apple Machine Learning Research Model License: LensVLM-9B may be used "exclusively for Research Purposes", and the licence excludes "any commercial exploitation, product development or use in any commercial product or service"
• Qwen Research License: Qwen-Image-2.1 is for non-commercial use only
• CC BY-NC 4.0 and gated access: Cohere Labs’ tiny-aya-l2-thinker requires accepting conditions to download and forbids commercial use

How to run them on your machine with llama.cpp

None of the month’s small models is in the official Ollama library: their pages returned 404 on 28 September. The direct route is llama.cpp with a GGUF from Hugging Face. These are the commands I used for MiMo, with its vision projector:

mkdir -p modelos && cd modelos
repo=https://huggingface.co/bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF
curl -LO "$repo/resolve/main/MiMo-V2.6-Distill-Qwen-9B-Q4_K_M.gguf"
curl -LO "$repo/resolve/main/mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf"

The first download is the quantised weights and the second the image encoder. ggml-org’s GGUF only exists in Q8_0 (9.53 GB), which is why I used bartowski’s Q4_K_M quantisation. Then start the server in Docker:

docker run -d --name llama -p 127.0.0.1:8080:8080 \
  -v "$PWD":/models:ro \
  ghcr.io/ggml-org/llama.cpp:server-v0.5.0 \
  -m /models/MiMo-V2.6-Distill-Qwen-9B-Q4_K_M.gguf \
  --mmproj /models/mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf \
  --host 0.0.0.0 --port 8080 --jinja -c 8192

Run it from the same modelos directory.

The --jinja option enables the model’s chat template, without which tool calls do not come back as tool_calls. The web UI is at http://127.0.0.1:8080 and the OpenAI-compatible API at /v1/chat/completions. On my machine port 8080 was taken and I published 24101; the rest of the command is identical. If you prefer Ollama, the guide to importing a Hugging Face model into Ollama covers the same GGUF.

What to pick for the memory you have

The figure that matters is the size of the quantised file, plus whatever context cache you add on top. With that in mind:

Under 8 GB of RAM. MiniCPM5-2B in Q4_K_M takes 1.56 GB and generated 56.85 tokens/s on my CPU with 6 threads. Do not ask it for judgement or facts it cannot know, and always review the code it writes, but it handles simple tools. If you plan to use it in an agent, first check which open models handle tool calling well.

Between 8 and 16 GB. MiMo-V2.6-Distill-Qwen-9B in Q4_K_M is 5.84 GB of weights plus 0.92 GB if you load vision. It generates 18.03 tokens/s on a CPU, enough for an agent that is not in a hurry. It is my pick of the month for agentic tasks and reading screenshots, always with an external check on what it claims about options and versions.

32 GB or more. Nex-N2.5-mini in Q4_K_M, 22.32 GB, is the candidate to try: being an MoE it generated 25.15 tokens/s, more than MiMo. If you come from dense models, first read how quantisation affects quality in llama.cpp, because in an MoE going down to Q3 or IQ2 changes more than the size.

Frequently asked questions

Why are Qwen, Gemma and Llama missing from the September review?

Because they published no new text models that month. I checked by listing their Hugging Face organisations by creation date on 28 September 2026. Qwen only uploaded image models, Google nothing for text, and Meta has not yet published the Muse Spark 1.2 weights.

Is MiMo-V2.6-Distill reliable for an agent that touches my server?

For choosing tools and filling in their arguments, yes in my tests. For stating which options a program has, no: it invented --auto-heal in all five attempts with the default sampling and mixed real and invented options in the same answer. Validate any command it proposes before running it, or give it a tool that looks up the documentation.

Does temperature 0 guarantee repeatable answers in llama.cpp?

Not entirely. With two or more simultaneous requests, llama.cpp processes them as a batch and the rounding changes, so the same question can get a different answer. It happened to me with MiMo’s Python function: the first version failed all seven cases and the second passed six.

Conclusion

September 2026 leaves a single release that changes anything for a home machine: MiMo-V2.6-Distill-Qwen-9B. It is a 9B with vision and an MIT licence that in my tests chose tools well and returned clean JSON. It also leaves a lesson no model-card table teaches: the same model that rejects a non-existent option at temperature 0 invents it with the default sampling.

Download it, use it for agentic work, and do not trust a command-line option it gives you without checking. The Spanish version of this article is at Qué modelos abiertos de septiembre de 2026 merece la pena ejecutar.

Sources: [1] Xiaomi MiMo, MiMo-V2.6-Distill-Qwen-9B model card^[1], [2] bartowski, MiMo-V2.6-Distill-Qwen-9B GGUF^[2], [3] OpenBMB, MiniCPM5-2B model card^[3], [4] Danish Foundation Models, DFM Mimir v1.5 model card^[4], [5] llama.cpp v0.5.0 release notes^[5], [6] Nex-AGI, Nex-N2.5-mini model card^[6], [7] DeepSeek, DeepSeek-V4.1-Flash model card^[7], [8] DeepSeek, DeepSeek-V4-Flash-Vision-Exp model card^[8], [9] InternLM, Intern-Decision-4B model card^[9], [10] Apple, LensVLM-9B licence^[10], [11] Microsoft, VibeVoice-ASR-Streaming-1.5B model card^[11], [12] docker run reference in the Docker documentation^[12].

Sources

1. Xiaomi MiMo, MiMo-V2.6-Distill-Qwen-9B model card
2. bartowski, MiMo-V2.6-Distill-Qwen-9B GGUF
3. OpenBMB, MiniCPM5-2B model card
4. Danish Foundation Models, DFM Mimir v1.5 model card
5. llama.cpp v0.5.0 release notes
6. Nex-AGI, Nex-N2.5-mini model card
7. DeepSeek, DeepSeek-V4.1-Flash model card
8. DeepSeek, DeepSeek-V4-Flash-Vision-Exp model card
9. InternLM, Intern-Decision-4B model card
10. Apple, LensVLM-9B licence
11. Microsoft, VibeVoice-ASR-Streaming-1.5B model card
12. docker run reference in the Docker documentation