| quant | needs | speed |
|---|---|---|
| ~ FP16 | 4.0 GB | ~1–2 tok/s (slow) |
| ✓ Q8_0 | 2.5 GB | ~19–30 tok/s (fast) |
| ✓ Q4_K_M | 1.8 GB | ~30–50 tok/s (fast) |
Same model, other machines: Apple M1 Pro · Apple M2 · Apple M3 Pro · Apple M2 Pro · Apple M4 Pro · Apple M3 Max
Also runs on a Apple M1: Qwen2.5 · Qwen3 · Llama 3.2 · Gemma 2 · Gemma 3
Best models for a Apple M1 · 8GB →
These share DeepSeek-R1 (distill)'s KV-cache geometry — they size memory the same way, so here's how they fit on a Apple M1 · 8GB too:
| model | shares | verdict here |
|---|---|---|
| Llama 3.1 | 8kv/128hd | 🐢 CPU-only (slow) |
| Qwen2.5-Coder | 2kv/128hd4kv/128hd8kv/128hd | ✅ Runs comfortably |
| Llama 3.3 | 8kv/128hd | ❌ Won't fit |
| Mistral 7B | 8kv/128hd | 🐢 CPU-only (slow) |
| Mistral Small 3 | 8kv/128hd | ❌ Won't fit |
| Mistral Nemo | 8kv/128hd | ❌ Won't fit |
| Codestral | 8kv/128hd | ❌ Won't fit |
| Llama 3.2 Vision | 8kv/128hd | ❌ Won't fit |
…and 12 more — see the DeepSeek-R1 (distill) page.
Check any combo yourself: open the checker →