Can I run TinyLlama?

1.1B — the classic 'will it run on a potato' model. Yes, it will.

chatgeneral · sizes: 1.1B · Apache-2.0

What runs it

machinebest fitverdictspeed
Apple M1 · 8GB 1.1B FP16 ✅ Runs (tight) 17–30 tok/s
Apple M1 Pro · 16GB 1.1B FP16 ✅ Runs comfortably 45–80 tok/s
Apple M2 · 16GB 1.1B FP16 ✅ Runs comfortably 25–45 tok/s
Apple M3 Pro · 18GB 1.1B FP16 ✅ Runs comfortably 35–60 tok/s
Apple M2 Pro · 32GB 1.1B FP16 ✅ Runs comfortably 45–80 tok/s
Apple M4 Pro · 48GB 1.1B FP16 ✅ Runs comfortably 60–100 tok/s
Apple M3 Max · 36GB 1.1B FP16 ✅ Runs comfortably 80–130 tok/s
Apple M2 Max · 64GB 1.1B FP16 ✅ Runs comfortably 80–130 tok/s
Apple M4 Max · 128GB 1.1B FP16 ✅ Runs comfortably 100–170 tok/s
Apple M2 Ultra · 192GB 1.1B FP16 ✅ Runs comfortably 110–180 tok/s
NVIDIA GeForce RTX 3060 · 12GB VRAM 1.1B FP16 ✅ Runs comfortably 65–110 tok/s
NVIDIA GeForce RTX 4070 · 12GB VRAM 1.1B FP16 ✅ Runs comfortably 90–150 tok/s
NVIDIA GeForce RTX 4090 · 24GB VRAM 1.1B FP16 ✅ Runs comfortably 180–310 tok/s
NVIDIA GeForce RTX 5080 · 16GB VRAM 1.1B FP16 ✅ Runs comfortably 180–290 tok/s
NVIDIA GeForce RTX 5090 · 32GB VRAM 1.1B FP16 ✅ Runs comfortably 330–540 tok/s
AMD Radeon RX 7900 XTX · 24GB VRAM 1.1B FP16 ✅ Runs comfortably 170–280 tok/s

Shopping for a machine? What hardware do I need to run TinyLlama? →

Check your exact rig: open the checker →

$npx runlocal-sh can-i-run tinyllama