Updated: 2026-07-07

Mistral AI released Mixtral 8x22B on April 10, 2024, with their characteristic style: a Twitter magnet link without a prior blog post or conference. The community downloaded the weights within hours, and benchmarks appeared the next day. It is the next generation of their MoE (Mixture of Experts) architecture, with 141B total parameters but only 39B active per forward pass. This changes the economics of serving open models.

Key takeaways

  • Mixtral 8x22B’s MoE architecture activates only 39B of its 141B parameters per token: large-model capability at medium-model inferential cost.

  • Apache 2.0 with no commercial restrictions: the most permissive large-scale option, alongside Llama 3.

  • Superior multilingual performance vs Llama 3 70B, especially in Spanish, French, Italian, and German.

  • Minimum hardware is an A100 80GB or H100 80GB to serve it quantised at Q4, and with little headroom; a 24 GB consumer GPU does not reach.

  • Self-hosting pays off if you sustain more than 100M tokens/month; below that, hosted services are more efficient.

What Mixtral 8x22B Is

The Sparse Mixture of Experts architecture:

  • 8 "experts" of 22B parameters each.

  • Router that selects 2 experts per token.

  • Total: 141B parameters on disk.

  • Active per forward pass: ~39B (2 experts plus shared components).

The result: ~141B capacity at ~39B inferential cost. A better quality-to-cost ratio than an equivalent dense model.

License and distribution

Apache 2.0. No commercial-use restrictions. Mistral AI first published the magnet link with no prior notice and, a week later, the official announcement[1] alongside the instruct version. The weights are available on Hugging Face (base[2] and instruct[3]), and the original magnet links still work.

Compared with Llama 3 70B (a more restrictive license) or Claude 3 (closed), Mixtral 8x22B is the most permissive large-scale option for commercial use with no user thresholds.

Key benchmarks

Benchmark Mixtral 8x22B Llama 3 70B GPT-4 GPT-3.5
MMLU 77.8 79.5 86.4 70.0
HellaSwag 88.9 88.0 95.3 85.5
GSM8K 78.6 93.0 92.0 57.1
HumanEval 45.1 81.7 88.4 48.1
Multilingual (FR, ES, IT, DE) Excellent Good Excellent Medium

These figures vary somewhat by evaluation harness; the ones in this table come from public comparisons aggregated after launch and from Mistral AI’s official announcement[1].

Superior multilingual performance vs Llama 3 70B, especially for European enterprise use. Behind on maths vs Llama 3 70B, and on coding vs Claude 3 Opus.

Required hardware: the limiting factor

Precision VRAM
FP16 ~280 GB
INT8 ~140 GB
INT4 (GGUF Q4_K_M) ~80 GB
INT3 ~60 GB

A 4090 (24 GB) cannot serve it even quantised. A single A100 80GB or H100 80GB is right at the INT4 limit, loading the model but leaving little headroom for context and batching; in practice, 2× A100/H100 80GB gives real headroom. Apple Silicon M3 Max 128GB reportedly runs Q4 at roughly 5-10 tokens/s.

Mixtral 8x22B vs Mixtral 8x7B

The smaller sibling (46.7B total, 12.9B active):

Aspect 8x7B 8x22B
Total parameters 46.7B 141B
Active/token 12.9B 39B
VRAM at Q4 ~25 GB ~80 GB
Overall quality ~GPT-3.5 Between GPT-3.5 and GPT-4
Multilingual Very good Excellent
Tokens/s (A100 Q4) ~60 ~25

For many use cases, 8x7B is the more pragmatic choice: faster, cheaper, and good enough. The 8x22B makes sense when quality matters more than throughput. Both are solid bases for quantisation with llama.cpp in edge deployments.

Production serving

With vLLM[4]:

# vLLM with tensor parallel
python -m vllm.entrypoints.openai.api_server 
  --model mistralai/Mixtral-8x22B-Instruct-v0.1 
  --tensor-parallel-size 2 
  --gpu-memory-utilization 0.9 
  --max-model-len 32768

With llama.cpp[5] for Q4:

./server -m mixtral-8x22b-instruct-Q4_K_M.gguf 
  -c 16384 -ngl 99 --host 0.0.0.0 --port 8080

vLLM gives the best GPU throughput. llama.cpp is more portable and handles mixed CPU-GPU offload.

Cost of serving

  • 1× A100 80GB on-prem: about $15k/year amortised.

  • AWS p4d.24xlarge (8× A100 40GB): between $22 and $33/hour depending on region and commitment (P4 instance pricing[6]), roughly $16k-$24k/month.

  • Together.ai hosted[7]: $1.20 per million input and output tokens.

Self-hosting pays off if you sustain more than 100M tokens/month. Below that, hosted services such as Together.ai, Anyscale, or Mistral’s La Plateforme are more efficient.

Real-world use cases

Mixtral 8x22B excels at:

  • Enterprise multilingual work: documents in ES/FR/IT/DE/EN.

  • RAG with long context (native 64k-token window).

  • Complex summarisation and analysis.

  • Self-hosting under strict compliance: no data leaving the premises.

Other models win at:

  • Maths: Llama 3 70B or Claude 3 Opus.

  • Top-tier coding: Claude 3 Opus, DeepSeek Coder.

  • Ultra-long context: Gemini 1.5.

Conclusion

Mixtral 8x22B confirms that Mistral AI leads the open frontier in Europe. Its MoE architecture strikes an attractive balance between quality and inferential efficiency. For teams that can afford the hardware, it is currently the best open option for serious multilingual use cases. For those who cannot, Mixtral 8x7B remains valid as the lighter option. And for serious production without dedicated GPUs, hosted services offer pay-per-token access. The open ecosystem keeps closing the gap with closed frontier models, a natural companion to the Claude 3 family for organisations that want an open-weight multilingual alternative.

Also read the Spanish version: Mixtral 8x22B: mixture of experts abierto y potente.

Sources

  1. the official announcement
  2. base
  3. instruct
  4. vLLM
  5. llama.cpp
  6. P4 instance pricing
  7. Together.ai hosted