How runlocal computes verdicts

Three invariants first: verdicts are free, rankings are commercially blind (no sponsorships, no affiliate re-sorting — ever), and when accuracy and reach conflict, accuracy wins. Every number below is computed by @runlocal/core at build time — the same engine behind the CLI, the checker, and all 592 matrix verdicts — so this page cannot drift from the code.

1 · The memory budget: usable, not total

The #1 lie in “will it fit” math is using total memory. On Apple Silicon, Metal caps GPU-wired memory (~67% of unified memory on ≤36GB machines, ~75% above) and the OS + your apps need headroom on top. On NVIDIA/AMD we reserve max(0.8 GiB, 10%) of VRAM for the CUDA context, fragmentation, and your compositor — and honor memory.free when known.

2 · Weights: measured bytes first, calibrated bpw as fallback

Where the catalog carries a measured GGUF file size, we use it directly. Across 96 measured quants, the naive flat bits-per-weight estimate is off by 2.0% on average — every data point is public on the accuracy leaderboard. Otherwise we fall back to per-quant calibrated bits-per-weight — K-quants carry mixed-precision overhead, which is why Q4_K_M is 4.85 bpw, not 4:

3 · KV cache: GQA-aware, or you’re off by 8×

The KV cache grows with context, and it’s where naive estimators go wrong: they key off attention heads or parameter count. Modern models use Grouped-Query Attention — far fewer key/value heads than attention heads — so we size from the KV-head count in the catalog:

kv = 2 (K,V) × layers × kv_heads × head_dim × context × bytes/elem (FP16 KV, Ollama’s default)

That last row is why other checkers wrongly fail long-context models that actually fit — counting every attention head overstates this KV cache by 8×.

4 · The verdict

weights + KV vs the usable budget → comfortable (fits with headroom), tight (fits — close other apps), partial-offload / CPU-only (runs, slowly, spilling past the accelerator), or won’t fit. Because KV grows with context, every verdict also carries a context curve (“comfortable to 32K · runs to 64K”) computed by sweeping the same fit function — so the curve can never contradict the verdict.

5 · Speed: a bandwidth-derived range, never a fake number

Decode is memory-bound: every generated token streams the active weights + KV through the memory bus, so tok/s ≈ MBU × bandwidth / bytes-per-token. MBU (memory-bandwidth utilization) is the fraction real runtimes actually achieve — calibrated against published llama.cpp/Ollama benchmarks, and bandwidth-dependent on Apple, where wide-bus Ultras achieve a lower fraction than base chips:

We publish a range, never a point estimate — an exact tok/s for hardware nobody measured is a guess with confident formatting. Field check: on a real M1 Pro (200 GB/s), llama3.2:3b measured 53.5 tok/s against our predicted 40–65 (mid ~53) — dead-center (2026-07-06).

6 · The loop stays closed

After every runlocal install, the CLI measures the real decode rate and prints it against the prediction — being wrong is visible immediately, on your machine, not just ours. The accuracy page tracks predicted-vs-measured publicly. Found a rig where we’re off? Tell us — accuracy is the product.