Cosmovex/tools

Local LLMs on a Mac with 16 GB: what an M4 can run and how fast

Mac with 16 GB with Local AI Fit

This page opens with an M4 and 16 GB of unified memory at 8K context. macOS reserves part of memory for the system, so the model gets about two thirds of it by default. The list shows what fits, the quantisation, and an estimated speed from the chip's memory bandwidth.

  1. Check the chip and memory at the top; pick your own chip to compare.
  2. Read which models fit, with the quantisation chosen.
  3. Tap Llama 3.1 8B to see the memory it needs against the 10.7 GB available.
  4. Raise the memory to 24 GB or 32 GB to see which models are added.
  5. Open "What do I need?" to see which Mac configurations hold a bigger model.
mac 16gb local llm m4 16gb llmmacbook air local llmollama mac 16gbunified memory llm
Open Local AI Fit Free to start · no account

What to know

By default macOS lets the GPU wire about two thirds of unified memory on machines up to 32 GB, which is about 10.7 GB on a 16 GB Mac. Llama 3.1 8B at Q8_0 needs about 9.6 GB at 8K context, so it fits, but with little left for the system and other apps. The estimate is about 9 tokens per second on an M4, whose memory bandwidth is 120 GB/s.

Smaller models feel much better. Llama 3.2 3B at Q8_0 runs near 23 tokens per second by this estimate and Llama 3.2 1B near 59. A 14B model at Q4_K_M needs about 10.3 GB, which technically fits but leaves almost nothing for macOS, so expect swapping and stalls in real use. 30B-class and larger models do not fit on a 16 GB Mac.

Apple silicon is attractive because its memory is shared: a 64 GB or 96 GB Mac can hold models that no single consumer graphics card can. A 16 GB machine is the entry point, good for small models and for trying things out. The reverse tab lists the Mac configurations that hold a given model.

Updated · Cosmovex

Questions

Can a 16 GB Mac run a 7B or 8B model?

Yes. An 8B model at Q4_K_M needs about 6 GB and runs comfortably; at Q8_0 it needs about 9.6 GB, which fits within the default GPU limit but leaves little for other apps.

Why is the usable memory less than 16 GB?

macOS keeps part of unified memory for the system. By default the GPU can use about two thirds on machines up to 32 GB and about three quarters above that. The limit can be raised with a system setting, but doing so risks instability.

Is an M4 faster than an M2 for LLMs?

By memory bandwidth, yes: 120 GB/s against 100 GB/s for the base chips, so about 20% faster by this estimate. Pro and Max chips have much more bandwidth (273 GB/s for M4 Pro, up to 546 GB/s for M4 Max) and are far faster.

Does prompt processing feel slower than generation on a Mac?

Often yes. Reading a long prompt is limited by compute rather than memory bandwidth, so long documents take longer to start than the tokens-per-second figure suggests. This tool estimates generation speed only.