Can I run TinyLlama?

1.1B — the classic 'will it run on a potato' model. Yes, it will.

chatgeneral · sizes: 1.1B · Apache-2.0

What runs it

machinebest fitverdictspeed
Apple M1 · 8GB 1.1B FP16 ✅ Runs (tight) 18–30 tok/s
Apple M1 Pro · 16GB 1.1B FP16 ✅ Runs comfortably 50–85 tok/s
Apple M2 · 16GB 1.1B FP16 ✅ Runs comfortably 30–45 tok/s
Apple M3 Pro · 18GB 1.1B FP16 ✅ Runs comfortably 40–65 tok/s
Apple M2 Pro · 32GB 1.1B FP16 ✅ Runs comfortably 50–85 tok/s
Apple M4 Pro · 48GB 1.1B FP16 ✅ Runs comfortably 65–110 tok/s
Apple M3 Max · 36GB 1.1B FP16 ✅ Runs comfortably 85–140 tok/s
Apple M2 Max · 64GB 1.1B FP16 ✅ Runs comfortably 85–140 tok/s
Apple M4 Max · 128GB 1.1B FP16 ✅ Runs comfortably 110–180 tok/s
Apple M2 Ultra · 192GB 1.1B FP16 ✅ Runs comfortably 150–250 tok/s
NVIDIA GeForce RTX 3060 · 12GB VRAM 1.1B FP16 ✅ Runs comfortably 70–120 tok/s
NVIDIA GeForce RTX 4070 · 12GB VRAM 1.1B FP16 ✅ Runs comfortably 100–170 tok/s
NVIDIA GeForce RTX 4090 · 24GB VRAM 1.1B FP16 ✅ Runs comfortably 200–330 tok/s
NVIDIA GeForce RTX 5080 · 16GB VRAM 1.1B FP16 ✅ Runs comfortably 190–310 tok/s
NVIDIA GeForce RTX 5090 · 32GB VRAM 1.1B FP16 ✅ Runs comfortably 350–590 tok/s
AMD Radeon RX 7900 XTX · 24GB VRAM 1.1B FP16 ✅ Runs comfortably 180–300 tok/s

Shopping for a machine? What hardware do I need to run TinyLlama? →

Check your exact rig: open the checker →

$npx runlocal-sh can-i-run tinyllama