You downloaded the model.
Now what?
Twenty gigabytes on disk, a card from 2017, and a VRAM calculator that told you it is not possible. It is. And the hard part is not starting it — it is keeping it running.
Why another guide
Guides exist. We read them. They all stop at the same place: „it loaded, it works". And almost all of them are written on 24 GB cards — that is, on hardware where anything runs anyway.
What they do not cover is what happens after six hours. When speed drops and you do not know why. When the cache is thrown away on every message. When a second person joins and everything halves. When the optimisation everybody recommends makes it worse.
The eleven chapters
8 pages, no filler. Every chapter carries at least one measured number, with the machine and the date next to it. MEASURED Every figure in the list below is our own measurement, taken between 24 August and 12 September 2026 on the two machines described in the guide.
- 1 · The VRAM calculator lied to you What an MoE is, why the experts can spill into system memory, and the command that starts a 35-billion-parameter model on an 11 GB card.
- 2 · „Does it fit" and „is it useful" are two questions The speed thresholds, and the formula for estimating on your own machine.
- 3 · One slot, not two Why `-np 2` makes speed jump between 6 and 30 seconds. With the numbers.
- 4 · What happens after six hours The CPU degradation table, hour by hour, and why it does not happen on a GPU. Plus the trap that nearly emptied the fix.
- 5 · Prefix cache on disk: 15 minutes down to half a second How to save and restore it, and two traps that had already cost us.
- 6 · The obvious optimisation that hurts +15% on the small case, −40% on the real one. Three AI systems recommended it.
- 7 · Threads are two questions, not one Prefill doubles, generation stays flat. The table for 4, 6, 8, 10 and 12 threads.
- 8 · Downloading the model: eleven times faster 10 MiB/s versus 116 MiB/s. Fifty minutes become four.
- 9 · When a second person joins Why the answer is not more slots, but a queue.
- 10 · Ready-made commands, by card class From „no GPU" up to 24 GB. Plus how to tune `--n-cpu-moe` step by step, and the table of what never to set, whatever your hardware.
- 11 · The checklist Eight lines to tick before you say it works.
Proof that we know what we are talking about
Read the first chapter, free
This is chapter 1 of 11, word for word as it is in the guide you would be buying — not a summary written for a sales page. If it is not worth €25 to you after reading it, it is not, and you keep your money.
1. The VRAM calculator lied to you
If you went to one of the seven VRAM calculators floating around and got „not possible",
remember one thing: they answer a single question, and they ask it wrong.
They compute: parameters × bits = how much memory you need. Does it fit?
For a 35-billion-parameter model at Q4 that is ~21 GB. Your card has 11. So: no.
And that is false, if the model is an MoE.
What an MoE is, in three lines
An ordinary („dense") model uses all its parameters for every token it writes. An MoE is split
into many „experts", and for each token only a few light up. In Qwen 35B-A3B, out of 35 billion
parameters, about 3 do the work on each token.
The part that always works — attention and the router, the thing that decides which expert fires —
is small. The huge part is the experts, and they are used rarely and one at a time.
So you do this
Put attention and the router on the card and leave the experts in system memory. For each
token, only the slice of expert that is needed gets read from RAM, not the whole model.
```
llama-server -m model.gguf --n-cpu-moe 999 -ngl 99 -np 1 -c 20480
```
--n-cpu-moe says how many experts stay on the CPU. Start with a large number to keep them all
there, then lower it until the card starts filling up.
Measured on the card, 30 August 2026: 23.7 tokens per second. On a card from 2017, with a
35-billion-parameter model that a VRAM calculator refuses you.
⭐ That is the whole move. The rest of this guide is about what happens after it starts.
---
How to get it
The guide and the files
The full text, the measurement tables and the ready-made commands.
We set it up on your machine
We connect, choose, tune, and you end up with something that starts by itself.