Cohere's RAG/tool-use specialist. No GQA, so KV cache is heavy at long context.
ragagentsgeneral · sizes: 35B · CC-BY-NC-4.0
| machine | best fit | verdict | speed |
|---|---|---|---|
| Apple M1 · 8GB | 35B Q3_K_M | ❌ Won't fit | — |
| Apple M1 Pro · 16GB | 35B Q3_K_M | ❌ Won't fit | — |
| Apple M2 · 16GB | 35B Q3_K_M | ❌ Won't fit | — |
| Apple M3 Pro · 18GB | 35B Q3_K_M | ❌ Won't fit | — |
| Apple M2 Pro · 32GB | 35B Q3_K_M | 🐢 CPU-only (slow) | 0.8–1 tok/s |
| Apple M4 Pro · 48GB | 35B Q4_K_M | ✅ Runs (tight) | 6–11 tok/s |
| Apple M3 Max · 36GB | 35B Q4_K_M | 🐢 CPU-only (slow) | 1–2 tok/s |
| Apple M2 Max · 64GB | 35B Q5_K_M | ✅ Runs comfortably | 7–12 tok/s |
| Apple M4 Max · 128GB | 35B Q8_0 | ✅ Runs comfortably | 6–10 tok/s |
| Apple M2 Ultra · 192GB | 35B Q8_0 | ✅ Runs comfortably | 9–15 tok/s |
| NVIDIA GeForce RTX 3060 · 12GB VRAM | 35B Q3_K_M | 🐢 CPU-only (slow) | — |
| NVIDIA GeForce RTX 4070 · 12GB VRAM | 35B Q3_K_M | 🐢 CPU-only (slow) | — |
| NVIDIA GeForce RTX 4090 · 24GB VRAM | 35B Q4_K_M | ⚠️ Partial GPU offload | 6–10 tok/s |
| NVIDIA GeForce RTX 5080 · 16GB VRAM | 35B Q3_K_M | 🐢 CPU-only (slow) | — |
| NVIDIA GeForce RTX 5090 · 32GB VRAM | 35B Q3_K_M | ✅ Runs (tight) | 40–70 tok/s |
| AMD Radeon RX 7900 XTX · 24GB VRAM | 35B Q4_K_M | ⚠️ Partial GPU offload | 5–9 tok/s |
Shopping for a machine? What hardware do I need to run Command R? →
Check your exact rig: open the checker →