guide · old hardware

You downloaded the model.
Now what?

Twenty gigabytes on disk, a card from 2017, and a VRAM calculator that told you it is not possible. It is. And the hard part is not starting it — it is keeping it running.

€25 I want it First see what you can run

Why another guide

Guides exist. We read them. They all stop at the same place: „it loaded, it works". And almost all of them are written on 24 GB cards — that is, on hardware where anything runs anyway.

What they do not cover is what happens after six hours. When speed drops and you do not know why. When the cache is thrown away on every message. When a second person joins and everything halves. When the optimisation everybody recommends makes it worse.

The eleven chapters

Seven pages, no filler. Every chapter carries at least one measured number, with the machine and the date next to it. MEASURED  Every figure in the list below is our own measurement, taken between 24 August and 12 September 2026 on the two machines described in the guide.

• 1 · The VRAM calculator lied to you What an MoE is, why the experts can spill into system memory, and the command that starts a 35-billion-parameter model on an 11 GB card.
• 2 · „Does it fit" and „is it useful" are two questions The speed thresholds, and the formula for estimating on your own machine.
• 3 · One slot, not two Why `-np 2` makes speed jump between 6 and 30 seconds. With the numbers.
• 4 · What happens after six hours The CPU degradation table, hour by hour, and why it does not happen on a GPU. Plus the trap that nearly emptied the fix.
• 5 · Prefix cache on disk: 15 minutes down to half a second How to save and restore it, and two traps that had already cost us.
• 6 · The obvious optimisation that hurts +15% on the small case, −40% on the real one. Three AI systems recommended it.
• 7 · Threads are two questions, not one Prefill doubles, generation stays flat. The table for 4, 6, 8, 10 and 12 threads.
• 8 · Downloading the model: eleven times faster 10 MiB/s versus 116 MiB/s. Fifty minutes become four.
• 9 · When a second person joins Why the answer is not more slots, but a queue.
• 10 · Ready-made commands, by card class From „no GPU" up to 24 GB. Plus how to tune `--n-cpu-moe` step by step, and the table of what never to set, whatever your hardware.
• 11 · The checklist Eight lines to tick before you say it works.

Proof that we know what we are talking about

How to get it