Local LLM benchmarks on Apple Silicon

Latest measurement: 27 October 2026. Full reference run: 15 September 2026.

This page collects the local LLM speeds we have measured on an Apple M5 Max with 128 GB of unified memory, served with oMLX. Every figure comes from a measurement record with its date, version and configuration, and all of them are in a CSV you can download and reuse under CC BY 4.0.

On oMLX 0.7.3.dev2 with Lightning MTP, Qwen3.6-35B-A3B oQ4e-mtp generates code at 147.2 tokens/s and Qwen3.8-27B oQ4e-mtp at 70.8 tokens/s. On oMLX 0.6.4, in September, they gave 119.2 and 39.7 tokens/s. They are medians of 6 answers of 768 tokens requested through the API, the way an application would request them.

Download the data (CSV)

What the dataset contains

The CSV has one row per measurement: model, configuration, method, prompt length, number of repeats, median, minimum and maximum, and the record it comes from. It holds everything the charts show plus a few more measurements, such as the total time of the long-context runs.

Decode: tokens generated per second once the answer starts. It is the speed you see while reading.
TTFT: time to first token. With long prompts, prefill dominates it.
Prefill: prompt tokens processed per second.
Full answer: seconds from sending the request to the last token.
Peak memory: the highest MLX memory oMLX records during the run, model weights included.

Latest measurement: oMLX 0.7.3.dev2 (27 October 2026)

The October check repeats September’s code-answer method with the three models the author uses every day, all with Lightning MTP. Qwen3.8-27B is 79 % faster and Qwen3.6-35B-A3B 23 % faster.

Decode speed on code answers, September against October tokens/s · higher is better

Qwen3.6-35B-A3B oQ4e-mtp

15 Sep · oMLX 0.6.4: 119.2

27 Oct · oMLX 0.7.3.dev2: 147.2

Qwen3.8-27B oQ4e-mtp

15 Sep · oMLX 0.6.4: 39.7

27 Oct · oMLX 0.7.3.dev2: 70.8

Gemma 4 26B-A4B it oQ4e-mtp

15 Sep · oMLX 0.6.4: -

27 Oct · oMLX 0.7.3.dev2: 126.2

Show the data table

Model (Lightning MTP)	15 Sep · oMLX 0.6.4	27 Oct · oMLX 0.7.3.dev2
Qwen3.6-35B-A3B oQ4e-mtp	119.2	147.2
Qwen3.8-27B oQ4e-mtp	39.7	70.8
Gemma 4 26B-A4B it oQ4e-mtp	-	126.2

Apple M5 Max, 128 GB. Median of 6 answers of 768 tokens through the API (streaming /v1/chat/completions, temperature 0.6, thinking off). Gemma 4 26B-A4B was not measured in September.

Time to complete a 768-token code answer s · lower is better

Qwen3.6-35B-A3B oQ4e-mtp

15 Sep · oMLX 0.6.4: 6.9 s

27 Oct · oMLX 0.7.3.dev2: 5.4 s

Qwen3.8-27B oQ4e-mtp

15 Sep · oMLX 0.6.4: 20.4 s

27 Oct · oMLX 0.7.3.dev2: 11.2 s

Gemma 4 26B-A4B it oQ4e-mtp

15 Sep · oMLX 0.6.4: -

27 Oct · oMLX 0.7.3.dev2: 6.2 s

Show the data table

Model (Lightning MTP)	15 Sep · oMLX 0.6.4	27 Oct · oMLX 0.7.3.dev2
Qwen3.6-35B-A3B oQ4e-mtp	6.9 s	5.4 s
Qwen3.8-27B oQ4e-mtp	20.4 s	11.2 s
Gemma 4 26B-A4B it oQ4e-mtp	-	6.2 s

Apple M5 Max, 128 GB. Median of 6 answers, from sending the request to the last token.

The difference cannot be put down to the oMLX version alone. In September the Mac sat at thermal pressure Heavy almost the whole time, and in October thermal pressure was not recorded. mlx, mlx-lm and mlx-vlm also changed between the two dates. What stayed the same: the model files, the three coding prompts, the temperature and the answer length.

Full reference through the API: oMLX 0.6.4 (15 September 2026)

On code answers, the speculative acceleration that suits each model gave between 1.19 and 1.88 times the base speed. With 15,000 tokens of context the gain fell to between 1.07 and 1.47 times, and no configuration generated more slowly than its base.

Decode on code answers, with and without acceleration tokens/s · higher is better

Qwen3.6-35B-A3B 8-bit (VLM MTP)

No acceleration: 79.7

With acceleration: 117.5

Qwen3.6-35B-A3B oQ4e-mtp (Lightning MTP)

No acceleration: 100.2

With acceleration: 119.2

Qwen3.8-27B MLX-4bit (VLM MTP)

No acceleration: 24.1

With acceleration: 34.5

Qwen3.8-27B oQ4e-mtp (Lightning MTP)

No acceleration: 21.1

With acceleration: 39.7

Gemma 4 12B it 8-bit (VLM MTP)

No acceleration: 27.4

With acceleration: 50.5

Show the data table

Model (acceleration)	No acceleration	With acceleration
Qwen3.6-35B-A3B 8-bit (VLM MTP)	79.7	117.5
Qwen3.6-35B-A3B oQ4e-mtp (Lightning MTP)	100.2	119.2
Qwen3.8-27B MLX-4bit (VLM MTP)	24.1	34.5
Qwen3.8-27B oQ4e-mtp (Lightning MTP)	21.1	39.7
Gemma 4 12B it 8-bit (VLM MTP)	27.4	50.5

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Median of 6 answers of 768 tokens (three coding prompts × 2) through the API, temperature 0.6.

Time to complete a 768-token code answer, with and without acceleration s · lower is better

Qwen3.6-35B-A3B 8-bit (VLM MTP)

No acceleration: 10.0 s

With acceleration: 6.9 s

Qwen3.6-35B-A3B oQ4e-mtp (Lightning MTP)

No acceleration: 8.0 s

With acceleration: 6.9 s

Qwen3.8-27B MLX-4bit (VLM MTP)

No acceleration: 32.6 s

With acceleration: 22.8 s

Qwen3.8-27B oQ4e-mtp (Lightning MTP)

No acceleration: 37.3 s

With acceleration: 20.4 s

Gemma 4 12B it 8-bit (VLM MTP)

No acceleration: 28.9 s

With acceleration: 15.9 s

Show the data table

Model (acceleration)	No acceleration	With acceleration
Qwen3.6-35B-A3B 8-bit (VLM MTP)	10.0 s	6.9 s
Qwen3.6-35B-A3B oQ4e-mtp (Lightning MTP)	8.0 s	6.9 s
Qwen3.8-27B MLX-4bit (VLM MTP)	32.6 s	22.8 s
Qwen3.8-27B oQ4e-mtp (Lightning MTP)	37.3 s	20.4 s
Gemma 4 12B it 8-bit (VLM MTP)	28.9 s	15.9 s

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Median of 6 answers.

The long-context runs ask two questions about a Python module of about 15,100 tokens (17,024 with Gemma’s tokenizer) and request 512 tokens of answer.

Decode with about 15,000 tokens of context, with and without acceleration tokens/s · higher is better

Qwen3.6-35B-A3B 8-bit (VLM MTP)

No acceleration: 84.0

With acceleration: 90.7

Qwen3.6-35B-A3B oQ4e-mtp (Lightning MTP)

No acceleration: 100.2

With acceleration: 115.5

Qwen3.8-27B MLX-4bit (VLM MTP)

No acceleration: 21.7

With acceleration: 23.2

Qwen3.8-27B oQ4e-mtp (Lightning MTP)

No acceleration: 21.4

With acceleration: 27.2

Gemma 4 12B it 8-bit (VLM MTP)

No acceleration: 26.6

With acceleration: 39.2

Show the data table

Model (acceleration)	No acceleration	With acceleration
Qwen3.6-35B-A3B 8-bit (VLM MTP)	84.0	90.7
Qwen3.6-35B-A3B oQ4e-mtp (Lightning MTP)	100.2	115.5
Qwen3.8-27B MLX-4bit (VLM MTP)	21.7	23.2
Qwen3.8-27B oQ4e-mtp (Lightning MTP)	21.4	27.2
Gemma 4 12B it 8-bit (VLM MTP)	26.6	39.2

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Median of 2 answers of 512 tokens, temperature 0.6.

Time to first token with about 15,000 tokens of context s · lower is better

Qwen3.6-35B-A3B 8-bit (VLM MTP)

No acceleration: 6.2 s

With acceleration: 6.0 s

Qwen3.6-35B-A3B oQ4e-mtp (Lightning MTP)

No acceleration: 5.8 s

With acceleration: 6.1 s

Qwen3.8-27B MLX-4bit (VLM MTP)

No acceleration: 34.5 s

With acceleration: 32.6 s

Qwen3.8-27B oQ4e-mtp (Lightning MTP)

No acceleration: 31.3 s

With acceleration: 35.2 s

Gemma 4 12B it 8-bit (VLM MTP)

No acceleration: 20.1 s

With acceleration: 19.3 s

Show the data table

Model (acceleration)	No acceleration	With acceleration
Qwen3.6-35B-A3B 8-bit (VLM MTP)	6.2 s	6.0 s
Qwen3.6-35B-A3B oQ4e-mtp (Lightning MTP)	5.8 s	6.1 s
Qwen3.8-27B MLX-4bit (VLM MTP)	34.5 s	32.6 s
Qwen3.8-27B oQ4e-mtp (Lightning MTP)	31.3 s	35.2 s
Gemma 4 12B it 8-bit (VLM MTP)	20.1 s	19.3 s

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Median of 2 requests; prompt prefill dominates this time.

oMLX built-in benchmark by prompt length (oMLX 0.6.4)

The built-in benchmark measures in-process, with no HTTP, no chat template and greedy sampling (temperature 0), so it shows the best case for speculative decoding. DFlash was the fastest option with 1,000-token prompts on three of the five models, but with 32,000 tokens it fell far below the base on all four Qwen models. On Gemma 4 12B it stayed above.

One warning before comparing rows: this benchmark loads vision models on the text engine unless Lightning MTP or VLM MTP is on. That is why the no-acceleration row of Qwen3.6-35B-A3B oQ4e-mtp comes out much slower here than through the API.

Qwen3.6-35B-A3B 8-bit (mlx-community): decode by prompt length tokens/s · higher is better

No acceleration

1k: 92.1

4k: 91.1

16k: 84.1

32k: 77.9

VLM MTP

1k: 118.3

4k: 106.6

16k: 100.2

32k: 94.4

DFlash (w4 draft)

1k: 112.4

4k: 110.5

16k: 32.6

32k: 16.6

Show the data table

Configuration	1k	4k	16k	32k
No acceleration	92.1	91.1	84.1	77.9
VLM MTP	118.3	106.6	100.2	94.4
DFlash (w4 draft)	112.4	110.5	32.6	16.6

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Built-in benchmark, median of 3 repeats, 256 generated tokens, code_python profile. Drafters: Qwen3.6-35B-A3B-MTP-bf16 (VLM MTP) and z-lab/Qwen3.6-35B-A3B-DFlash quantised to 4 bits.

Qwen3.6-35B-A3B oQ4e-mtp (Jundot): decode by prompt length tokens/s · higher is better

No acceleration

1k: 49.3

4k: 46.1

16k: 45.2

32k: 42.2

Lightning MTP

1k: 77.2

4k: 85.8

16k: 87.9

32k: 82.8

DFlash (w4 draft)

1k: 142.7

4k: 86.3

16k: 39.2

32k: 17.3

Show the data table

Configuration	1k	4k	16k	32k
No acceleration	49.3	46.1	45.2	42.2
Lightning MTP	77.2	85.8	87.9	82.8
DFlash (w4 draft)	142.7	86.3	39.2	17.3

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Built-in benchmark, median of 3 repeats. The no-acceleration row runs on the text engine (see the warning above).

Qwen3.8-27B MLX-4bit (lmstudio-community): decode by prompt length tokens/s · higher is better

No acceleration

1k: 30.6

4k: 27.3

16k: 25.0

32k: 23.5

VLM MTP

1k: 39.3

4k: 36.6

16k: 36.4

32k: 30.9

DFlash2 (w4 draft)

1k: 48.2

4k: 43.7

16k: 35.8

32k: 6.3

DFlash2 (bf16 draft)

1k: 36.9

4k: 35.1

16k: 22.3

32k: 5.9

Show the data table

Configuration	1k	4k	16k	32k
No acceleration	30.6	27.3	25.0	23.5
VLM MTP	39.3	36.6	36.4	30.9
DFlash2 (w4 draft)	48.2	43.7	35.8	6.3
DFlash2 (bf16 draft)	36.9	35.1	22.3	5.9

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Built-in benchmark, median of 3 repeats. Drafters: Qwen3.8-27B-MTP-bf16 (VLM MTP) and z-lab/Qwen3.8-27B-DFlash2.

Qwen3.8-27B oQ4e-mtp (Jundot): decode by prompt length tokens/s · higher is better

No acceleration

1k: 27.0

4k: 26.4

16k: 24.4

32k: 21.4

Lightning MTP

1k: 48.0

4k: 37.7

16k: 41.1

32k: 35.4

Lightning MTP + TurboQuant KV 4-bit

1k: 42.2

4k: 37.7

16k: 39.7

32k: 30.3

Lightning MTP + tuned ANE prefill

1k: 35.6

4k: 33.1

16k: 35.8

32k: 30.9

DFlash2 (w4 draft)

1k: 41.4

4k: 33.8

16k: 18.8

32k: 5.8

Show the data table

Configuration	1k	4k	16k	32k
No acceleration	27.0	26.4	24.4	21.4
Lightning MTP	48.0	37.7	41.1	35.4
Lightning MTP + TurboQuant KV 4-bit	42.2	37.7	39.7	30.3
Lightning MTP + tuned ANE prefill	35.6	33.1	35.8	30.9
DFlash2 (w4 draft)	41.4	33.8	18.8	5.8

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Built-in benchmark, median of 3 repeats. DFlash2 drafter: z-lab/Qwen3.8-27B-DFlash2 at 4 bits.

Gemma 4 12B it 8-bit (mlx-community): decode by prompt length tokens/s · higher is better

No acceleration

1k: 31.0

4k: 30.5

16k: 30.1

32k: 28.8

VLM MTP

1k: 65.6

4k: 42.7

16k: 50.6

32k: 49.0

DFlash (w4 draft)

1k: 92.4

4k: 34.4

16k: 53.4

32k: 51.2

Show the data table

Configuration	1k	4k	16k	32k
No acceleration	31.0	30.5	30.1	28.8
VLM MTP	65.6	42.7	50.6	49.0
DFlash (w4 draft)	92.4	34.4	53.4	51.2

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Built-in benchmark, median of 3 repeats. Drafters: gemma-4-12B-it-assistant-bf16 (VLM MTP) and z-lab/gemma4-12B-it-DFlash at 4 bits.

gpt-oss-20b MXFP4-Q8 (mlx-community): decode by prompt length tokens/s · higher is better

No acceleration, first batch (partly cold)

1k: 120.2

4k: 106.5

16k: 100.4

32k: 78.8

No acceleration, hot repeat

1k: 126.6

4k: 121.1

16k: 80.8

32k: 69.2

Show the data table

Batch	1k	4k	16k	32k
No acceleration, first batch (partly cold)	120.2	106.5	100.4	78.8
No acceleration, hot repeat	126.6	121.1	80.8	69.2

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Built-in benchmark, median of 3 repeats. The first batch opened the session with the Mac partly cold; the repeat ran at the end, hot.

Prefill without acceleration by prompt length tokens/s · higher is better

Qwen3.6-35B-A3B 8-bit

1k: 3,440

4k: 2,426

16k: 2,450

32k: 2,162

Qwen3.6-35B-A3B oQ4e-mtp

1k: 3,133

4k: 2,258

16k: 1,986

32k: 1,701

Qwen3.8-27B MLX-4bit

1k: 913

4k: 525

16k: 546

32k: 500

Qwen3.8-27B oQ4e-mtp

1k: 629

4k: 545

16k: 545

32k: 480

Gemma 4 12B it 8-bit

1k: 1,150

4k: 955

16k: 985

32k: 899

gpt-oss-20b MXFP4-Q8

1k: 3,744

4k: 3,737

16k: 1,755

32k: 1,622

Show the data table

Model	1k	4k	16k	32k
Qwen3.6-35B-A3B 8-bit	3,440	2,426	2,450	2,162
Qwen3.6-35B-A3B oQ4e-mtp	3,133	2,258	1,986	1,701
Qwen3.8-27B MLX-4bit	913	525	546	500
Qwen3.8-27B oQ4e-mtp	629	545	545	480
Gemma 4 12B it 8-bit	1,150	955	985	899
gpt-oss-20b MXFP4-Q8	3,744	3,737	1,755	1,622

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Built-in benchmark, median of 3 repeats; gpt-oss-20b, hot repeat.

Peak memory without acceleration up to 32,000 prompt tokens GiB

Qwen3.6-35B-A3B 8-bit

38.7 GiB

Qwen3.6-35B-A3B oQ4e-mtp

23.2 GiB

Qwen3.8-27B MLX-4bit

21.7 GiB

Qwen3.8-27B oQ4e-mtp

22.2 GiB

Gemma 4 12B it 8-bit

14.5 GiB

gpt-oss-20b MXFP4-Q8

12.4 GiB

Show the data table

Model	Peak
Qwen3.6-35B-A3B 8-bit	38.7 GiB
Qwen3.6-35B-A3B oQ4e-mtp	23.2 GiB
Qwen3.8-27B MLX-4bit	21.7 GiB
Qwen3.8-27B oQ4e-mtp	22.2 GiB
Gemma 4 12B it 8-bit	14.5 GiB
gpt-oss-20b MXFP4-Q8	12.4 GiB

Apple M5 Max, 128 GB, oMLX 0.6.4, 15 September 2026. Highest MLX memory the built-in benchmark records, weights included.

Hardware and versions

Item	15 September 2026	27 October 2026
Machine	Apple M5 Max, 40-core GPU, 128 GB of unified memory	The same
oMLX	0.6.4, menu bar app	0.7.3.dev2
Engines	Not recorded	mlx 0.32.4, mlx-lm 0.32.3.dev8, mlx-vlm 0.7.10
Thermal pressure	Heavy in 76 of the 80 built-in benchmark cells	Not recorded
Per-model settings	One per run, as each chart states	Lightning MTP with adaptive depth of up to 3 tokens; TurboQuant, DFlash, ANE prefill and SpecPrefill off

The Mac is the author’s work machine. In September it had other applications open, with about 48 GB in use before any model loaded. The absolute figures are those of a Mac in real use, not of a dedicated machine running cold.

Method

We use two methods, and figures from one are not compared with figures from the other.

Through the API: streaming requests to /v1/chat/completions from a container that runs on the Mac itself (OrbStack), with temperature 0.6, top_p 0.95 and thinking off. Each request carries a unique prefix so the prefix cache cannot help. The code answers are three prompts (Python, TypeScript and Go) repeated twice, with 768 tokens of answer, after one warm-up request. Decode is generated tokens minus one, divided by the time from the first to the last chunk received.
Built-in benchmark: oMLX’s POST /admin/api/bench/start with prompts of 1,025, 4,097, 16,385 and 32,769 tokens, 256 generated tokens, the code_python profile and align_prompt_to_ane on, which also keeps the results from being uploaded to omlx.ai. We ran it three times per configuration and report the median.

Comparisons with and without acceleration always use the same engine on both sides. The September runs took one session of about three hours, changing one setting at a time. When each acceleration pays off, and why, is in the guide to installing and tuning oMLX on the M5 Max.

What these figures do not say: they do not measure answer quality, they are not a model ranking, and on another chip or with another amount of memory the numbers change. To estimate other hardware, use the local LLM calculator.

Updates

We measure again with each new oMLX release that changes the minor number (0.8, 0.9…), with the same method. Earlier measurements stay in the CSV with their date and version, so a citation to a given date stays valid.

How to cite

The data (the CSV and the figures on this page) are published under the Creative Commons Attribution 4.0 International licence (CC BY 4.0). You may copy, adapt and publish them, commercially too, as long as you credit the source and link to this page. State the date of the measurement you use, because the figures change with every version.

Javier Cañete (2026), Local LLM benchmarks on Apple Silicon [dataset], jacar.es, https://jacar.es/en/local-llm-benchmarks-apple-silicon/, measurement of 27 October 2026, CC BY 4.0 licence.

@misc{jacar_llm_apple_silicon_2026,
  author = {Cañete, Javier},
  title  = {Local LLM benchmarks on Apple Silicon},
  year   = {2026},
  url    = {https://jacar.es/en/local-llm-benchmarks-apple-silicon/},
  note   = {Dataset, CC BY 4.0. Measurement of 27 October 2026}
}