Gemma 4 26B on a 24 GB Mac mini M4:
from 66% CPU to 100% GPU

TL;DR

Three facts and one command.

Bench — Mac mini M4 · 24 GB unified memory · macOS 26.5 · Ollama 0.34.0 · gemma4:26b Q4_K_M (MoE, 25.2B total / ~4B active) · measured 2026-09-13

sudo sysctl iogpu.wired_limit_mb=20480      # raise the Metal wired-memory ceiling
# restart Ollama, then:
ollama ps                                   # PROCESSOR must read "100% GPU"

Why it is slow out of the box

Unified memory is not the same as "the GPU can use all 24 GB".

macOS limits how much of unified memory the GPU may pin ("wire"). On a 24 GB machine the default ceiling is about 17.8 GiB. Ollama measures this as its available GPU memory. gemma4:26b in Q4_K_M projects to about 18.1 GB on-device once you add a 64K context, the vision projector and the MTP draft model used for speculative decoding. It does not fit, so Ollama's fitter (common_params_fit_impl) keeps roughly 3 GB of headroom and moves the mixture-of-experts weights of 9 layers to the CPU.

Why does a CPU spill hurt so much on a MoE model? Generation is memory-bandwidth-bound and, for gemma4:26b, only ~4B parameters are active per token, which is why it can stream at dense-4B speeds. But when the expert tensors of 9 layers live on the CPU, every token does a CPU matmul plus a CPU↔GPU sync. You are paying the MoE routing cost without the MoE speed.

Diagnose your own machine

Two commands and two log lines tell you everything.

ollama ps
# NAME         SIZE    PROCESSOR          CONTEXT
# gemma4:26b   4.0 GB  66%/34% CPU/GPU    65536      ← anything but "100% GPU" is the problem

sysctl iogpu.wired_limit_mb
# iogpu.wired_limit_mb: 0                            ← 0 = macOS default (~2/3 of RAM on a 24 GB M4)

grep -E "MTL0 \(Apple M4\)|CPU_Mapped|MTL0_Mapped|gpu memory" ~/.ollama/logs/server.log | tail -5
# gpu memory ... library=Metal available="17.3 GiB" free="17.8 GiB"
# load_tensors:  MTL0_Mapped model buffer size =  8798.74 MiB
# load_tensors:   CPU_Mapped model buffer size =  5376.00 MiB   ← expert weights on CPU
# common_params_fit_impl: - MTL0 (Apple M4): 31 layers ( 9 overflowing), 14706 MiB used, 3479 MiB free

The fix

Raise the ceiling, persist it, then (and only then) tune the rest.

1. Raise the Metal wired limit (needs admin)

   20 GiB leaves about 3.5–4 GB for macOS and your editor. It is applied instantly and lost on reboot, so also install a LaunchDaemon.

   sudo sysctl iogpu.wired_limit_mb=20480
   
   sudo tee /Library/LaunchDaemons/com.minipod.iogpu-wired-limit.plist >/dev/null <<'EOF'
   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
   <plist version="1.0"><dict>
     <key>Label</key><string>com.minipod.iogpu-wired-limit</string>
     <key>ProgramArguments</key><array>
       <string>/usr/sbin/sysctl</string><string>iogpu.wired_limit_mb=20480</string>
     </array>
     <key>RunAtLoad</key><true/>
   </dict></plist>
   EOF
   sudo chown root:wheel /Library/LaunchDaemons/com.minipod.iogpu-wired-limit.plist
   sudo launchctl bootstrap system /Library/LaunchDaemons/com.minipod.iogpu-wired-limit.plist

2. Give Ollama.app its environment via launchd

   Ollama.app on macOS does not read ~/.zshrc. Use launchctl setenv from a login-time LaunchAgent. Gate the KV-cache quantization on the ceiling actually being raised (see the pitfall below for why).

   # ~/mini-pod/scripts/login-env.sh  (run by a LaunchAgent at login)
   launchctl setenv OLLAMA_KEEP_ALIVE 24h            # 26B cold load is ~20 s; keep it resident
   launchctl setenv OLLAMA_MAX_LOADED_MODELS 1        # 24 GB: one model at a time
   launchctl setenv OLLAMA_NUM_PARALLEL 1             # single user = fastest
   launchctl setenv OLLAMA_FLASH_ATTENTION 1
   
   if [[ "$(sysctl -n iogpu.wired_limit_mb)" -ge 20000 ]]; then
     launchctl setenv OLLAMA_KV_CACHE_TYPE q8_0       # halves KV memory; never q4_0 (recall degrades)
   else
     launchctl unsetenv OLLAMA_KV_CACHE_TYPE
   fi
   
   pgrep -xq Ollama || open -a Ollama                 # start Ollama *after* env is set

3. Set the context length where Ollama.app actually reads it

   OLLAMA_CONTEXT_LENGTH is silently overridden by the app's own setting stored in SQLite. 32K is enough for agent turns and shortens the worst-case prefill; set it while Ollama is stopped.

   sqlite3 "$HOME/Library/Application Support/Ollama/db.sqlite" \
     "update settings set context_length=32768 where id=1;"

4. Restart and verify

   pkill -x Ollama; pkill -f "Ollama.app/Contents/Resources/ollama"; sleep 3; open -a Ollama
   curl -s localhost:11434/api/generate -d '{"model":"gemma4:26b","prompt":"hi","stream":false}' >/dev/null
   ollama ps                                                   # → 100% GPU
   grep -c kIOGPUCommandBufferCallbackErrorOutOfMemory ~/.ollama/logs/server.log   # → 0
   sysctl kern.memorystatus_vm_pressure_level                  # 1 ok · 2 warn · 4 critical

The pitfall: tuning the "safe" knobs first breaks the model

We tried q8_0 KV cache and a 32K context before touching the ceiling. Every request then failed.

Intuition says a smaller context and a smaller KV cache can only help. In practice the fitter saw the freed memory and kept more expert layers on the GPU (9 → 6 overflowing). Its projection does not fully account for Metal heap overhead, the vision projector and the MTP draft model, so at decode time the GPU ran out of wired memory:

ggml_metal_synchronize: error: command buffer 0 failed with status 5
error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
ggml_metal_graph_compute: backend is in error state from a previous command buffer failure
srv  update_slots: decode() failed: Compute error.

Results

Same machine, same model file, same prompts, /api/generate timings.

What you should not expect: cloud-class speed. M4 (non-Pro) has roughly 120 GB/s of memory bandwidth, and generation on a 4B-active MoE tops out around 35–40 tok/s on that budget. 23 tok/s with a 32K context, flash attention and speculative decoding is a realistic place to land on a 24 GB machine that is also running an editor.

Surviving a reboot

Three things have to happen in order, and nothing does it for you by default.

Two details we tripped on: Ollama.app registers an "open at login" item via SMAppService, but on this machine it was disabled, so nothing started Ollama after a reboot at all. And two copies of the same LaunchAgent had accumulated; harmless, but delete duplicates so there is one source of truth.

Post-reboot check, in one command: ./scripts/status.sh — it prints the wired limit, the daemon's last exit code, the KV type, the app's context length, the fitter's overflow line, the Metal OOM count and the memory pressure level.

FAQ / edge cases

Memory pressure shows level 2 (warn) after the change. Is that ok?

Yes if the box is a dedicated inference node. macOS + an editor have ~3.5 GB. If it reaches level 4 (critical) or the machine starts swapping hard, drop the ceiling one notch: sudo sysctl iogpu.wired_limit_mb=19456. The fitter will overflow one layer and you keep most of the speed.

Still seeing 1–2 overflowing layers at 20480?

Ollama wants roughly 3.1–3.5 GB of headroom. If your model blob or context is a bit larger than ours, go to 21504. Do not go higher on 24 GB.

Why not OLLAMA_NUM_GPU=99 like the older guides say?

Ollama 0.34 already offloads every layer it can (offloaded 43/43 layers to GPU in the log). The spill is inside the layers (expert tensors), decided by the fitter, and driven by the memory ceiling. Layer count is not the lever.

Does OLLAMA_KV_CACHE_TYPE=q4_0 buy more?

A few hundred MB, at a real cost in long-context recall. Attention re-reads the cache thousands of times; keep it at q8_0.

Does OLLAMA_CONTEXT_LENGTH do anything with Ollama.app?

Not when the app has its own value in settings.context_length; the server log will show the app's value under server config. Edit the SQLite row (with Ollama stopped) or use the app's UI.

Should I raise OLLAMA_NUM_PARALLEL?

No. Each parallel slot multiplies the KV cache and compute buffers. Single-agent use is fastest at 1.

Prefill is my bottleneck (long agent prompts). What helps?

The ceiling does not; prefill is compute-bound. Keep the system prompt byte-stable so Ollama's prompt cache hits, keep tool output out of the transcript where you can, and cap context at 32K. Every 10K tokens of prompt costs ~40 s on this chip regardless of GPU placement.