Can I run Mistral 7B?

It is a dense model: all 7 billion parameters work on every token, so it has to fit in the card's memory to run at a usable speed. At Q4 the file is about 4 GB. Below is every graphics card we track, whether Mistral 7B runs on it with 32 GB of DDR4-3200 system RAM, and how fast — with a label on every number saying whether we measured it or computed it.

39 / 39cards run it with 32 GB of RAM

39 / 39with 16 GB

4 GBsmallest card that runs it

⭐ What it is good at, from our own use: rapid, cerinte mici

On every card, with 32 GB of DDR4-3200 system RAM

card	runs?	speed	how we know
No graphics card (CPU only)	runs	about 3 tok/s	ESTIMATED from memory bandwidth
GTX 1050 Ti	slow but works	3–4 tok/s	ESTIMATED from memory bandwidth
GTX 970	slow but works	3–4 tok/s	ESTIMATED from memory bandwidth
GTX 1060 6GB	slow but works	6–8 tok/s	ESTIMATED from memory bandwidth
GTX 980 Ti	slow but works	8–11 tok/s	ESTIMATED from memory bandwidth
RTX 2060	slow but works	8–11 tok/s	ESTIMATED from memory bandwidth
GTX 1660 Super	slow but works	8–11 tok/s	ESTIMATED from memory bandwidth
RTX 3060 Laptop 6GB	slow but works	8–11 tok/s	ESTIMATED from memory bandwidth
RTX 3050 8GB	runs	31–39 tok/s	ESTIMATED from memory bandwidth
GTX 1070	runs	35–44 tok/s	ESTIMATED from memory bandwidth
RTX 4060	runs	38–47 tok/s	ESTIMATED from memory bandwidth
RTX 4060 Laptop 8GB	runs	38–47 tok/s	ESTIMATED from memory bandwidth
GTX 1080	runs	44–55 tok/s	ESTIMATED from memory bandwidth
RTX 2070	runs	62–77 tok/s	ESTIMATED from memory bandwidth
RTX 2060 Super	runs	62–77 tok/s	ESTIMATED from memory bandwidth
RTX 3070	runs	62–77 tok/s	ESTIMATED from memory bandwidth
RTX 3080 10GB	runs	105–120 tok/s	ESTIMATED from memory bandwidth
GTX 1080 Ti	runs	67–83 tok/s	ESTIMATED from memory bandwidth
RTX 2080 Ti	runs	85–106 tok/s	ESTIMATED from memory bandwidth
RTX 3060 12GB	runs	50–62 tok/s	ESTIMATED from memory bandwidth
RTX 4070	runs	70–87 tok/s	ESTIMATED from memory bandwidth
RTX 5070 12GB	runs	93–116 tok/s	ESTIMATED from memory bandwidth
Tesla T4 16GB (server)	runs	44–55 tok/s	ESTIMATED from memory bandwidth
RTX 4080 16GB	runs	99–120 tok/s	ESTIMATED from memory bandwidth
RTX 5080 16GB	runs	about 120 tok/s	ESTIMATED from memory bandwidth
Tesla M40 24GB (server card, very cheap)	runs	40–50 tok/s	ESTIMATED from memory bandwidth
L4 24GB (server)	runs	41–52 tok/s	ESTIMATED from memory bandwidth
Tesla P40 24GB (server card, cheap)	runs	48–60 tok/s	ESTIMATED from memory bandwidth
RTX 3090	runs	about 120 tok/s	ESTIMATED from memory bandwidth
RTX 4090	runs	about 120 tok/s	ESTIMATED from memory bandwidth
Tesla V100 32GB (server)	runs	about 120 tok/s	ESTIMATED from memory bandwidth
RTX 5090 32GB	runs	about 120 tok/s	ESTIMATED from memory bandwidth
A100 40GB (server)	runs	about 120 tok/s	ESTIMATED from memory bandwidth
L40S 48GB (server)	runs	119–120 tok/s	ESTIMATED from memory bandwidth
A100 80GB PCIe (server)	runs	about 120 tok/s	ESTIMATED from memory bandwidth
A100 80GB SXM (server)	runs	about 120 tok/s	ESTIMATED from memory bandwidth
H100 80GB SXM (server)	runs	about 120 tok/s	ESTIMATED from memory bandwidth
H100 NVL 94GB (server)	runs	about 120 tok/s	ESTIMATED from memory bandwidth
H200 141GB (server)	runs	about 120 tok/s	ESTIMATED from memory bandwidth

Questions people actually ask

Can I run Mistral 7B on my computer?

With 32 GB of system RAM it runs on 39 of the 39 graphics cards we track — and even with no graphics card at all. The table below says which, how fast, and for the ones where it does not fit, why not. Every speed carries a label saying whether we measured it or computed it.

What is the smallest graphics card that runs Mistral 7B?

The card with the least VRAM on our list that still runs it is the GTX 1050 Ti (4 GB). Past that, for a model like this one, system RAM decides as much as the card does.

Does it need 16 GB or 32 GB of RAM?

With 16 GB of system RAM it runs on 39 cards; with 32 GB, on 39. The difference is what the extra RAM buys you for this model.

Where do these numbers come from?

From the same calculation our API answers with (`what_can_i_run` on our MCP endpoint), and a guard proves it says the same thing as the JavaScript on our calculator page. Rows labelled MEASURED ran on our own machines, with the machine and date written beside them; everything else says ESTIMATED and is given as a range.

Want it for your exact machine, with your memory speed and quantisation? Ask the calculator, or call what_can_i_run on our MCP endpoint — it is the same calculation.

Can I run Mistral 7B?

On every card, with 32 GB of DDR4-3200 system RAM

Questions people actually ask

Other models