Run gpt-oss 20B with Local AI Fit
This page opens on the reverse tab for gpt-oss 20B at Q4_K_M and 8K context. It shows the memory required, the 16 GB graphics cards and the Macs that hold it entirely, and how its mixture-of-experts design affects speed.
- Read the memory required at the top of the result.
- Compare with the quantisation and context you plan to use.
- Check the cards listed; they hold the whole model in VRAM.
- Check the Macs listed for the smallest unified memory that works.
- Open "Will it run?" and enter your own hardware to see where it lands.
What to know
gpt-oss 20B has about 20.9 billion parameters in total and uses about 3.6 billion of them for each token, because only a few of its experts are chosen per token. At Q4_K_M the weights are about 11.8 GB, the cache at 8K context about 0.4 GB, and the runtime 0.7 GB, for about 12.9 GB in total. A 16 GB card holds it with room for a longer context.
Because so little is read per token, speed is much higher than for a dense model of that size. The estimate on an RTX 5080 is about 200 tokens per second, and a 16 GB card with 448 GB/s of bandwidth is still near 90. Even when part of the model spills into system RAM, as on a 12 GB card, the estimate stays around 55 tokens per second, since only the chosen experts are read.
On a Mac the model needs 24 GB of unified memory with the default GPU limit. Memory capacity decides whether it runs; bandwidth decides how quickly. The model is published in a compact native 4-bit format, so the figures here, based on the generic Q4_K_M size, approximate the real file size.
Updated · Cosmovex
Questions
How much VRAM does gpt-oss 20B need?
About 13 GB at Q4_K_M with an 8K context by this calculation. A 16 GB card holds it entirely; a 12 GB card holds most of it and uses system RAM for the rest.
Why is a 20B model so fast?
It is a mixture-of-experts model. Only about 3.6 billion parameters are active for each token, so each token reads about as much data as a 3.6B dense model while keeping the knowledge of a larger one.
Can I run gpt-oss 20B on 8 GB of VRAM?
Partly. With 16 GB of system RAM about 58% of the model sits on the card and the estimate is about 31 tokens per second, which is usable. Real memory use can differ slightly from this estimate because of the compact published format.
Does it run on a CPU?
Yes. At a 3-bit quantisation it needs about 10.6 GB, fits in 16 GB of RAM, and is estimated at about 13 tokens per second on DDR4-3200, faster than a dense 8B model on the same machine.