Can I run SmolLM2?

Tiny but surprisingly capable. Runs on almost anything, even CPU.

chatgeneral · sizes: 1.7B · Apache-2.0

What runs it

machinebest fitverdictspeed
Apple M1 · 8GB 1.7B FP16 🐢 CPU-only (slow) 1–2 tok/s
Apple M1 Pro · 16GB 1.7B FP16 ✅ Runs comfortably 30–55 tok/s
Apple M2 · 16GB 1.7B FP16 ✅ Runs comfortably 18–30 tok/s
Apple M3 Pro · 18GB 1.7B FP16 ✅ Runs comfortably 25–40 tok/s
Apple M2 Pro · 32GB 1.7B FP16 ✅ Runs comfortably 30–55 tok/s
Apple M4 Pro · 48GB 1.7B FP16 ✅ Runs comfortably 40–70 tok/s
Apple M3 Max · 36GB 1.7B FP16 ✅ Runs comfortably 55–90 tok/s
Apple M2 Max · 64GB 1.7B FP16 ✅ Runs comfortably 55–90 tok/s
Apple M4 Max · 128GB 1.7B FP16 ✅ Runs comfortably 65–110 tok/s
Apple M2 Ultra · 192GB 1.7B FP16 ✅ Runs comfortably 95–160 tok/s
NVIDIA GeForce RTX 3060 · 12GB VRAM 1.7B FP16 ✅ Runs comfortably 45–75 tok/s
NVIDIA GeForce RTX 4070 · 12GB VRAM 1.7B FP16 ✅ Runs comfortably 60–100 tok/s
NVIDIA GeForce RTX 4090 · 24GB VRAM 1.7B FP16 ✅ Runs comfortably 120–210 tok/s
NVIDIA GeForce RTX 5080 · 16GB VRAM 1.7B FP16 ✅ Runs comfortably 120–200 tok/s
NVIDIA GeForce RTX 5090 · 32GB VRAM 1.7B FP16 ✅ Runs comfortably 220–370 tok/s
AMD Radeon RX 7900 XTX · 24GB VRAM 1.7B FP16 ✅ Runs comfortably 110–190 tok/s

Shopping for a machine? What hardware do I need to run SmolLM2? →

Check your exact rig: open the checker →

$npx runlocal-sh can-i-run smollm2