How to combine Qwen and Clef on oMLX so one writes and the other decides

Qwen writes and Clef decides: since oMLX 0.7.1.dev1 both come from the same server on your Mac. Qwen answers on /v1/chat/completions and Clef, Cloudflare’s open decision model built on Qwen, on /v1/systemone: it reads a text once and returns a choice with its probability, without generating text. I measured it on 13 October 2026 on a MacBook M5 Max with 128 GB, with 40 support tickets, a judge for context-grounded answers and the tweaks that worked and those that did not. Every latency includes the HTTP hop over the local network. There is also a Spanish version.

Key takeaways

• Clef-flash (9B, 5.86 GB) decided each ticket in a median of 111 ms and got 39 of 40 labels right; Clef (27B, 15.81 GB) took 371 ms and got all 40.
• Splitting the work beats asking Qwen for everything: Clef-flash labels and Qwen 9B writes, with 39 of 40 labels right against 34 when Qwen did both in one JSON.
• As a RAG judge, Clef 27B flagged all 4 answers Qwen 9B made up; Clef-flash only 2.
• Turning on Lightning MTP for Qwen 27B raised generation from 29.3 to 42.7 tokens/s (+46 %). Prefill on the Neural Engine did not move Clef beyond the noise.
• Pin the models in memory: the four take 42.2 GB, and pinning avoids cold loads and the 500 error in jundot/omlx#4413.

Why separate the model that writes from the one that decides

An LLM like Qwen generates text token by token, and if you ask it for a label it returns text you have to validate. A decision model scores the options you give it in a single read. Michelle Chen sums it up in Cloudflare’s announcement^[1]: "The decision step is non-autoregressive, so there’s no intermediate text to generate token by token".

Clef starts from a 27B Qwen base and Clef-flash from a 9B one, both under Apache 2.0. Instead of a sentence you get JSON with the chosen option, the probability of each alternative and a confidence. That lets you set a threshold and send doubtful cases to a person, which free text does not give you.

The combination makes sense because each model does what it is best at. Clef decides the route (label, urgency, whether a person is needed) and Qwen only writes when the route calls for it. Afterwards, Clef reads the draft again to decide whether it stays within what the customer said.

What you need to serve Qwen and Clef from the same oMLX

You need oMLX 0.7.1.dev1 or later on a Mac with Apple Silicon. If oMLX is new to you, start with what oMLX is and how it differs from Ollama and the install guide on an M5 Max. Version 0.7.1.dev1 includes PR jundot/omlx#4315^[2], merged on 6 October 2026, which adds a decision model type with its own engine.

Before that change, Clef loaded like any vision model. The PR puts it this way: "Before this change oMLX loaded these checkpoints as plain VLMs and returned meaningless text." Now the dashboard marks it as DECISION and the chat API rejects it. I checked with a call to /v1/chat/completions against clef-flash-oQ4e:

HTTP 400
Model 'clef-flash-oQ4e' is not an LLM / chat model.
Use /v1/systemone for decision models.

These are the four models I used, with the size the oMLX dashboard shows:

How to call Clef on /v1/systemone

A request to /v1/systemone carries a state (the text being evaluated) and a map of questions. This one labels a ticket with one of four labels and, in the same call, asks whether the customer is blocked. The examples use a Spanish support queue, so the questions are in Spanish. If your server has an API key, add the Authorization: Bearer header, as the oMLX API key, port and endpoints guide explains:

curl -s http://localhost:8000/v1/systemone \
  -H "Content-Type: application/json" -d '{
  "model": "clef-flash-oQ4e",
  "state": "Al pulsar Exportar a PDF la app se cierra",
  "questions": {
    "etiqueta": {"type": "choice",
      "instructions": "¿Qué etiqueta corresponde al ticket?",
      "criteria": {"facturacion": "Cobros, facturas y reembolsos",
        "cuenta": "Acceso, contraseñas y usuarios",
        "error": "Fallos del software",
        "peticion": "Ideas y funciones nuevas"}},
    "urgente": {"type": "noul",
      "instructions": "¿El cliente no puede trabajar?"}}}'

The response contains the chosen option, the probability of each label and the confidence. For the noul question it returns a single number: the probability that the condition holds. The output_tokens field is 0, because Clef generates nothing:

{
  "model": "clef-flash-oQ4e",
  "answers": {
    "etiqueta": {
      "type": "choice",
      "choice": "error",
      "confidence": 0.9431,
      "probabilities": {
        "facturacion": 0.0236,
        "cuenta": 0.0082,
        "error": 0.9431,
        "peticion": 0.0251
      }
    },
    "urgente": {"type": "noul", "noul": 0.3582}
  },
  "usage": {"input_tokens": 278, "output_tokens": 0}
}

How fast and how accurate each model is on an M5 Max

Clef-flash is 3.3 times faster than Clef 27B and misses one more ticket out of 40. I measured each model on 40 hand-labelled tickets (28 in Spanish and 12 in English, 10 per label and 7 urgent), with two questions per call: the label and the urgency.

Clef-flash’s only miss was "Los acentos salen como símbolos raros en los informes exportados" (accents show up as odd symbols in exported reports), which it put under billing with a confidence of 0.59. The least confident hit sat at 0.67, so a 0.6 threshold would have sent the miss to a person.

Clef’s cost grows with the text it reads, not with what it returns. Prefill ran at about 2,600 tokens/s: 811 tokens took 0.32 s, 6,955 tokens 2.6 s and 16,384 tokens 6.4 s. Beyond 16,384 input tokens, oMLX truncates the text.

Qwen, on the other hand, is measured in generated tokens per second:

As a pure classifier, Qwen labels all 40 tickets correctly, but it judges urgency worse and is between 2.3 and 8 times slower than Clef-flash. For the server’s own methods and memory figures, see the oMLX benchmarks and memory ceiling.

Which optimisations worked and which did not

Two tweaks changed the numbers and one did not. I measured each with and without the change, on the same texts:

MTP had been off since the upgrade to 0.7.x and I only noticed by measuring, so check that switch in the model settings after every upgrade. Batching questions pays off because Clef reads the shared text once per request: five questions cost 209 ms, not five times 123.

Example 1: ticket triage with Clef-flash, Qwen and a reviewer

Triage uses all three roles: Clef-flash decides, Qwen 9B writes and Clef 27B reviews the draft before it goes out. All the code uses only the Python standard library. Save this minimal client as omlx_min.py; it reads the URL from OMLX_URL and, if set, the key from OMLX_API_KEY:

import json, os, urllib.request

URL = os.environ.get("OMLX_URL", "http://localhost:8000")
KEY = os.environ.get("OMLX_API_KEY")
HEAD = {"Content-Type": "application/json"}
if KEY:
    HEAD["Authorization"] = f"Bearer {KEY}"

def post(path, body):
    req = urllib.request.Request(URL + path, headers=HEAD,
                                 data=json.dumps(body).encode())
    with urllib.request.urlopen(req, timeout=300) as r:
        return json.load(r)

def decide(model, state, questions):
    body = {"model": model, "state": state, "questions": questions}
    return post("/v1/systemone", body)["answers"]

def chat(model, system, user, max_tokens=160, **extra):
    msgs = [{"role": "system", "content": system},
            {"role": "user", "content": user}]
    r = post("/v1/chat/completions", {
        "model": model, "messages": msgs, "max_tokens": max_tokens,
        "chat_template_kwargs": {"enable_thinking": False}, **extra})
    return r["choices"][0]["message"]["content"].strip()

The triage questions go in triaje.py. The persona question decides whether the ticket needs someone on your team, and its wording matters, as you will see below:

import sys
from pprint import pprint
from omlx_min import decide, chat

CLEF, REVISOR = "clef-flash-oQ4e", "clef-oQ4e"
QWEN = "Qwen3.8-9B-mlx-4Bit"
PREGUNTAS = {
    "etiqueta": {"type": "choice",
        "instructions": "¿Qué etiqueta corresponde al ticket?",
        "criteria": {"facturacion": "Cobros, facturas y reembolsos",
                     "cuenta": "Acceso, contraseñas y usuarios",
                     "error": "Fallos del software",
                     "peticion": "Ideas y funciones nuevas"}},
    "persona": {"type": "noul", "instructions":
        "¿El cliente está enfadado, amenaza con acciones legales"
        " o pide algo que solo puede decidir una persona?"},
    "idioma": {"type": "choice",
        "instructions": "¿En qué idioma está escrito?",
        "criteria": {"es": "Español", "en": "Inglés"}},
}

The reviewer gets the ticket and the draft together and answers two questions: whether the draft states facts the ticket does not give, and whether it states how the product works:

REVISION = {
    "datos": {"type": "noul", "instructions": "¿La RESPUESTA afirma"
        " hechos concretos (fechas, importes, plazos, que algo ya se"
        " ha hecho) que el TICKET no da?"},
    "producto": {"type": "noul", "instructions": "¿La RESPUESTA"
        " afirma cómo funciona el producto, qué permite o qué planes"
        " hay, sin que el TICKET lo diga?"}}

The function ties the three calls together. If confidence drops below 0.6 or Clef thinks a person is needed, the ticket never reaches Qwen. If the reviewer gives 0.5 or more to either of its questions, the draft goes to review instead of out:

def triaje(ticket):
    d = decide(CLEF, ticket, PREGUNTAS)
    etiqueta, conf = d["etiqueta"]["choice"], d["etiqueta"]["confidence"]
    if conf < 0.6 or d["persona"]["noul"] >= 0.5:
        return {"ruta": "persona", "etiqueta": etiqueta}
    idioma = "español" if d["idioma"]["choice"] == "es" else "inglés"
    borrador = chat(QWEN, f"Eres el soporte de una aplicación. Responde"
        f" en {idioma}, en 2 o 3 frases. Ticket de {etiqueta}. No"
        " inventes fechas, importes, plazos ni cómo funciona el"
        " producto. Si falta un dato, pídeselo al cliente.", ticket)
    r = decide(REVISOR, f"TICKET: {ticket}\n\nRESPUESTA: {borrador}",
               REVISION)
    p = max(r["datos"]["noul"], r["producto"]["noul"])
    return {"ruta": "revisar" if p >= 0.5 else "enviar",
            "etiqueta": etiqueta, "confianza": round(conf, 3),
            "inventa": round(p, 3), "borrador": borrador}

pprint(triaje(" ".join(sys.argv[1:])), width=72)

With OMLX_URL pointing at the Mac, this was the result for a software bug and for an angry customer:

$ python3 triaje.py \
    "Al guardar un cliente nuevo da error 500 y no se guarda nada"
{'borrador': 'Detecté un error 500 al guardar el cliente, pero para '
             'ayudarte necesito el número exacto del ticket, el '
             'dispositivo que usas, la información que intentaste '
             'ingresar y el mensaje de error completo que muestra la '
             'aplicación. Con esos datos podré investigar si es un '
             'problema de la plataforma, una falta de permisos o un '
             'requisito incompleto.',
 'confianza': 0.912,
 'etiqueta': 'error',
 'inventa': 0.08,
 'ruta': 'enviar'}
$ python3 triaje.py \
    "Me habéis cobrado dos veces y como no lo arregléis hoy os denuncio"
{'etiqueta': 'facturacion', 'ruta': 'persona'}

The ticket about a public API shows why the reviewer is there. Qwen does not write at temperature 0 in this example, so each run gives a different draft. In an earlier run it wrote "our roadmap does not include plans to release a public API", a product claim the ticket does not support, and Clef 27B gave it 0.871: review. In this one, the draft avoids promising anything and passed with 0.256:

$ python3 triaje.py "Any plans for a public API with webhooks?"
{'borrador': 'I am currently unable to provide details about future '
             'API or webhook roadmaps as that information is not '
             'available in the support system. Please direct inquiries '
             'regarding product development plans to our product team '
             'or community forums. If you have a specific technical '
             'implementation question about the current API, feel free '
             'to ask.',
 'confianza': 0.936,
 'etiqueta': 'peticion',
 'inventa': 0.256,
 'ruta': 'enviar'}

Over the 40 tickets, the combination got 39 labels right against 34 when Qwen 9B labelled and replied in the same JSON. The end-to-end median was 0.86 s against 0.79 s. Asking Qwen for both at once makes it label worse: labelling alone it got 40 of 40.

The wording of persona was the most profitable tweak. With "anger, legal, personal data, anything ambiguous", Clef sent 6 correctly labelled tickets (4 billing and 2 account) to review, with confidences from 0.77 to 0.98. With the current wording, which names what really needs a person, none. Thirty-nine went out on their own and the only one sent to review was the mislabelled ticket, with a confidence of 0.57.

For the reviewer, size matters. Even with the "don’t make things up" instruction, Qwen 9B stopped inventing dates and amounts but kept stating things about the product. Over 39 drafts, Clef-flash flagged 1 and Clef 27B flagged 5, in about 0.5 s per draft. Among them were one stating that the app cannot promote a teammate to administrator (0.90) and another denying plans for a public API (0.82).

Example 2: Clef 27B as a judge of context-grounded answers

In retrieval-augmented generation (RAG), the risk is that the model answers with something the context does not say. Here Qwen answers with a returns policy as context and Clef 27B decides whether the answer stays within it. If not, Qwen is asked for a second answer that only uses the context. Save it as juez.py next to omlx_min.py:

import sys
from pprint import pprint
from omlx_min import decide, chat

CLEF, QWEN = "clef-oQ4e", "Qwen3.8-9B-mlx-4Bit"
CONTEXTO = ("Política de devoluciones de Tienda Acme: el cliente"
    " puede devolver un producto en 30 días desde la entrega. El"
    " reembolso se hace al mismo medio de pago en un máximo de 14"
    " días. Los productos personalizados no admiten devolución. El"
    " envío de la devolución es gratuito en España peninsular; en"
    " Canarias cuesta 6 €.")
JUEZ = {"apoyada": {"type": "noul", "instructions":
    "¿Todo lo que afirma la RESPUESTA está respaldado por el CONTEXTO?"}}
SOLO = ("Responde en una frase usando SOLO el contexto. Si no lo dice,"
        " dilo así y ofrece pasar la consulta a una persona.")

apoyo() is the only call to Clef: it joins context, question and answer into the state and returns the probability that everything is supported. atender() sends the first answer if it scores above 0.5 and otherwise asks for the second one:

def responder(pregunta, sistema):
    return chat(QWEN, sistema, f"Contexto: {CONTEXTO}\n\n"
                f"Pregunta: {pregunta}", max_tokens=90, temperature=0)

def apoyo(pregunta, respuesta):
    estado = (f"CONTEXTO: {CONTEXTO}\n\nPREGUNTA: {pregunta}\n\n"
              f"RESPUESTA: {respuesta}")
    return round(decide(CLEF, estado, JUEZ)["apoyada"]["noul"], 3)

def atender(pregunta):
    primera = responder(pregunta, "Responde en una frase, como"
                        " atención al cliente de Tienda Acme.")
    p = apoyo(pregunta, primera)
    if p >= 0.5:
        return {"respuesta": primera, "apoyada": p}
    segunda = responder(pregunta, SOLO)
    return {"primera": primera, "apoyada_primera": p,
            "respuesta": segunda, "apoyada": apoyo(pregunta, segunda)}

pprint(atender(" ".join(sys.argv[1:])), width=72)

These were two runs, one with a question the context does not answer and one it does:

$ python3 juez.py "¿Hacéis cambios por otra talla?"
{'apoyada': 0.766,
 'apoyada_primera': 0.266,
 'primera': 'Sí, podemos gestionar el cambio de talla siempre que el '
            'producto no sea personalizado y se encuentre dentro del '
            'plazo de 30 días desde la entrega.',
 'respuesta': 'No se menciona en el contexto si hacéis cambios por '
              'otra talla.'}
$ python3 juez.py "¿Cuántos días tengo para devolver un producto?"
{'apoyada': 0.99,
 'respuesta': 'Tiene 30 días desde la entrega para devolver el '
              'producto.'}

The first answer about sizes (exchanges for another size) was a plausible invention, and the judge gave it 0.266. The second admits the context does not say, although Qwen did not offer to pass the query to a person as the instruction asked.

In the full test I used 10 questions: 5 are answered by the context and 5 are not. In 4 of the 5 that are not, Qwen 9B made the answer up: shops in Barcelona, size exchanges, returning items bought in a shop and the cost from Portugal. The fifth, "the refund is not in cash", does follow from the context. This is how the two judges scored:

Clef 27B also flagged an answerable reply that added "after we receive the product at our warehouse", which the context does not say. It is an embellishment, but a strict judge should flag it. If you already measure your RAG, this judge fits as one more metric in evaluating a RAG system.

What happens when Qwen and Clef work at the same time

Both models share the GPU, and when they overlap, Clef loses more than Qwen. With Qwen 27B generating at the same time, Clef-flash went from 125 to 243 ms per decision, because oMLX yields the GPU to it between prefill chunks. Qwen lost 6 %, from 28.5 to 26.9 tokens/s.

Memory is not the problem: the four models add up to 42.2 GB, below the roughly 75 GB ceiling oMLX keeps on this machine. The problem is cold loads and a bug in 0.7.1.dev1. If oMLX evicts the decision model while it is encoding a request, /v1/systemone returns a 500, according to issue jundot/omlx#4413^[3]. The fix is already in the repository, but it lands after 0.7.1.dev1.

The answer in both cases is to pin the models from each model’s settings in the dashboard. Model management in oMLX explains how to combine pinning with TTL and usage-based eviction.

Gotchas when running Qwen and Clef on the same server

These are the details that cost me a run or a wrong result:

• Turn off Qwen’s thinking: pass "chat_template_kwargs": {"enable_thinking": false} on every chat call, so Qwen does not spend tokens reasoning before it writes two sentences
• Write the instructions as what you want to detect: "ambiguous" or "personal data" trigger needless escalations; "asks for something only a person can decide" does not
• Keep the state short: every 2,600 tokens of text add a second, and above 16,384 tokens oMLX truncates it without an error
• Use the right model for each role: Clef-flash to decide and Clef 27B to review, because the small reviewer let through 4 of the 5 drafts the large one flagged
• Check MTP after every upgrade: after moving to 0.7.x it was off and Qwen 27B ran 46 % slower

Frequently asked questions

Can I use Clef as a chat model in oMLX?

No. Since 0.7.1.dev1, a call to /v1/chat/completions with a Clef model returns a 400 that points you to /v1/systemone. In earlier versions it loaded as a regular vision model and returned meaningless text.

Clef-flash or Clef 27B?

Clef-flash for inline decisions: 111 ms and 39 of 40 labels in 5.86 GB. Clef 27B for reviewing and judging, where mistakes cost more: it caught all 4 inventions in the test RAG against 2, at the price of 371 ms and 15.81 GB.

How much memory do I need to keep all four models loaded?

The dashboard adds them up to 42.2 GB: 5.86 for Clef-flash, 15.81 for Clef, 15.83 for Qwen 27B and 4.71 for Qwen 9B, plus each request’s cache. On this 128 GB Mac they stay below oMLX’s roughly 75 GB ceiling. With less memory, start with Clef-flash and Qwen 9B, which take 10.6 GB together.

Conclusion

Qwen and Clef on one oMLX split the work cleanly: Clef decides in 111 ms with probabilities you can threshold, and Qwen only writes when it has to. In my tests that split raised triage from 34 to 39 correct labels, and Clef 27B as a reviewer caught the inventions Qwen 9B slipped in despite the instruction not to make things up.

Next, pin the four models, turn on MTP for Qwen 27B and try the triage on 50 real tickets from your queue before you let it reply on its own.

Sources: [1] Introducing Clef, Cloudflare blog^[1], [2] jundot/omlx#4315, decision models on /v1/systemone^[2], [3] jundot/omlx#4413, 500 error when the decision model is evicted^[3], [4] Cloudflare/clef-flash on Hugging Face^[4], [5] Cloudflare/clef on Hugging Face^[5], [6] mlx-community/clef-flash-8bit on Hugging Face^[6].

Sources

1. Cloudflare’s announcement
2. PR jundot/omlx#4315
3. issue jundot/omlx#4413
4. Cloudflare/clef-flash on Hugging Face
5. Cloudflare/clef on Hugging Face
6. mlx-community/clef-flash-8bit on Hugging Face