Gemma 4 locally, which size fits your GPU

You have a specific graphics card and a specific question: which Gemma 4 size fits. Google shipped the family on 2 April 2026 in five sizes under Apache 2.0, and the weights are only half the sum. Here are the memory figures for each size, the real Ollama tags, and what the context window costs, because that is usually what blows the budget.

Key takeaways

• There are five sizes, not four: E2B, E4B, 12B Unified, 26B A4B and 31B Dense.
• The E in E2B and E4B means effective parameters; the A in 26B A4B means active ones.
• The 256K context belongs to the 12B, 26B and 31B. E2B and E4B stop at 128K.
• At 4 bits the 31B weighs 20 GB in Ollama, and its full window adds about 10.8 GiB of cache.
• The licence is Apache 2.0, replacing the bespoke terms that governed Gemma 1, 2 and 3.

What Gemma 4 is and what those letters mean

Gemma 4 is Google DeepMind’s family of open-weight models, multimodal on input (text and image everywhere, plus audio on the E2B, E4B and 12B) and text-only on output. The official card lists five variants: E2B, E4B, 12B Unified, 26B A4B and 31B Dense.

The letters are not decoration. On the E prefix, the model card says it plainly: "The ‘E’ in E2B and E4B stands for ‘effective’ parameters". The two small models use Per-Layer Embeddings, per-layer embedding tables that are huge but only serve quick lookups, so the effective count sits far below the total. The E2B declares 2.3B effective against 5.1B with embeddings; the E4B, 4.5B against 8B.

The A in 26B A4B is a different thing: active parameters. It is a mixture-of-experts model with 25.2B total, of which it activates 3.8B per token (8 experts out of 128, plus one shared). It runs about as fast as a 4B model but occupies memory like a 25B one, because every expert has to be resident. That distinction shows up on the VRAM bill, so settle it before pulling anything.

How much memory each size takes

Google publishes its own inference memory table. It builds in 20% loading overhead, but warns these are "the memory required to load the static model weights" and that they exclude both supporting software and the context window. Next to those figures I put the real size of the GGUF Ollama serves, which does not always agree.

The surprise is at the top. The generic gemma4:e2b tag weighs 7.2 GB, two and a half times Google’s own 4-bit estimate, because those per-layer embedding tables are not dropped to lower precision in the GGUF. The quantisation-aware build, gemma4:e2b-it-qat, comes in at 4.3 GB. Same story on the E4B: 9.6 GB on the plain tag against 6.1 GB on the QAT one. On the small models, pulling the default tag is an expensive habit.

Which tag to pull in Ollama

The Ollama library publishes 50 gemma4 tags and has racked up 23.8 million downloads. Google’s own documentation gives the shorthand: gemma4:e2b, gemma4:e4b, gemma4:26b and gemma4:31b.

ollama pull gemma4:e2b-it-qat       # laptop with no GPU, or an 8 GB card
ollama pull gemma4:12b-it-qat       # 12 GB card
ollama pull gemma4:26b-a4b-it-qat   # 24 GB: the MoE, fast, 256K window
ollama pull gemma4:31b-it-qat       # the most that fits in 24 GB, trimmed

OLLAMA_CONTEXT_LENGTH=32768 ollama serve
ollama ps                           # check CONTEXT and the GPU/CPU split

On an Apple Silicon Mac there are equivalent MLX tags (gemma4:31b-mlx, 19 GB) that use the native engine instead of llama.cpp. If you do not have the runtime yet, there is the Ollama install guide and the Apple Silicon version. And if the quantisation acronyms read as noise, model quantization and llama.cpp explains what each level buys you.

What the context window really costs

This is where the memory goes. Weights are a fixed number; the key-value cache grows with every token you feed in. Gemma 4 interleaves local attention layers with global ones, and the model card explains the trick: "To optimize memory for long contexts, global layers feature unified Keys and Values, and apply Proportional RoPE (p-RoPE)".

The arithmetic comes straight from the published 31B config. It has 60 layers: 50 with a 1024-token sliding window and 10 global. The global ones use 4 key-value heads of dimension 512, with key and value unified, in 16 bits. That is 10 layers times 4 heads times 512 times 2 bytes: 40 KiB per token.

The 50 sliding layers contribute a fixed 800 MiB or so, because they never hold more than their 1024-token window. Add the 20 GB of weights and the 31B at 256K asks for around 31 GB: a 24 GB card does not get there. At 32K it does fit, at roughly 22 GB total, with very little room to spare.

One detail that catches people out: Ollama does not open the whole window on its own. Its documentation sets the default context from available memory, 4K below 24 GiB of VRAM, 32K between 24 and 48, and 256K above that. Unless you set OLLAMA_CONTEXT_LENGTH, the headline 256K does not exist on your machine.

How good is it in Spanish

Google claims out-of-the-box support for more than 35 languages and pre-training on over 140. It publishes no Spanish-specific score, so treat any concrete figure on that front with suspicion. What does exist is MMMLU, the multilingual MMLU: 88.4% on the 31B, 86.3% on the 26B, 83.4% on the 12B, 76.6% on the E4B and 67.4% on the E2B, against 70.7% for Gemma 3 27B.

In practice: the two big ones write correct Spanish and hold register; the E2B comes apart in long sentences and drags in constructions that read as translated. If the job is writing Spanish, the step from E4B to 12B is more noticeable than any other step in the family.

What changed since Gemma 2 and Gemma 3

The card itself includes a column for Gemma 3 27B with thinking off, and the comparison is blunt. MMLU Pro: 85.2% for the 31B against 67.6%. AIME 2026 without tools: 89.2% against 20.8%. LiveCodeBench v6: 80.0% against 29.1%. Even the E4B, with 4.5B effective parameters, beats the previous generation’s 27B at coding (52.0% against 29.1%).

The structural changes are the configurable thinking mode, native function calling, a proper system role, and the long window. If you are coming from the 2024 generation, Gemma 2, Google’s open model is where this family starts.

The licence: Apache 2.0 instead of the Gemma terms

This is the least photogenic change and the one that matters most if you are building something commercial. Gemma 1, 2 and 3 shipped under the Gemma Terms of Use, an agreement written by Google. It required you to propagate the use restrictions as an enforceable clause to any third party you redistributed the model or a derivative to, incorporated a prohibited-use policy by reference, and reserved Google the right to restrict usage, remotely included.

Gemma 4 ships under Apache 2.0. The old terms page says so without ceremony: "For Gemma 4 terms, see the Gemma 4 license", and that licence is plain Apache 2.0. The restriction propagation and the prohibited-use policy disappear as contractual obligations, leaving a standard permissive licence you already know how to read. For a product, the gap between reviewing a bespoke agreement and applying Apache 2.0 is measured in billable hours.

Against Qwen and Llama in the same slot

The direct rival on a 24 GB card is Qwen3.8-27B: its qwen3.8:27b tag in q4_K_M weighs 18 GB with a 256K window, also Apache 2.0, and lands two gigabytes cheaper than Gemma’s 31B. Gemma’s edge is the rungs Qwen does not cover: the 26B A4B, which runs almost as fast as a 4B, and the E2B/E4B pair for machines with no GPU at all.

Llama 4 is simply not in this league. Its smallest tag, llama4:scout, takes 67 GB in q4_K_M. There is no consumer rung. If you want the rest of the recent crop, there is the open models of August 2026 round-up, and to fit any of them into a tool-using workflow, open models with tool calling.

Verdict by machine

Frequently asked questions

Can I run the 31B on a 24 GB card?

Yes, with the window trimmed. The QAT tag’s weights take 19 GB and at 32K tokens the cache adds about 2 GiB, so it fits with little air. At 256K it is roughly 31 GB and it does not.

Is the 26B A4B worth it over the 31B?

Depends what hurts. The 26B takes 3 GB less and answers much faster, because it only activates 3.8B parameters per token. The 31B wins on every metric, but the MMLU Pro gap is 2.6 points. For interactive use the MoE is usually the better buy.

Do I need an NVIDIA GPU?

No. Ollama publishes MLX tags for Apple Silicon at the same sizes, and the E2B and E4B variants work on CPU at a tolerable crawl. What decides it is the unified memory or the VRAM available, not the brand.

Conclusion

Picking a Gemma 4 size is an accounting problem, not a taste one. Add the weights of the tag you are about to pull, add the cache for the context you will actually use, and compare that against your memory. The QAT tags save between one and three gigabytes over the generic ones and ask for nothing in return, and the 26B A4B is the sweet spot of the whole family on a 24 GB card. The Spanish version of this article is at Gemma 4 en local, qué variante cabe en tu GPU.

Sources

1. Google DeepMind, Gemma 4 model card
2. Google, Gemma 4 inference memory table
3. Google, Gemma 4 announcement
4. Google, Gemma Terms of Use
5. Ollama, gemma4 tags
6. Ollama, default context length
7. Hugging Face, google/gemma-4-31B-it