Best Local LLM by Hardware Tier

The best local LLM is the largest good open-weights model whose file fits your memory with room left for context. On 16 GB, that is gpt-oss-20b. On a 24 GB card, a 4-bit Gemma 4 31B or a 30B-class coding model. On one 80 GB GPU, gpt-oss-120b. File sizes and licences below come from our own pages, checked 31 October 2026.

The short answer by hardware tier

These are starting points, not winners. Model families change quickly; the library pages show what is current. We have not run these models ourselves for this page.

How to think about "best"

Local models are not ranked by one test. Three things decide fit:

1. Memory. If the file does not fit, nothing else matters.
2. Job. Chat, autocomplete and agent work need different strengths. An agent needs tool calling, long context and the patience to run many turns.
3. Licence. Apache-2.0 and MIT carry no conditions that matter for most products. Custom licences, such as Llama's gated one or Qwen's community licence on some models, need reading.

The best open weights on the Artificial Analysis table we read on 31 October 2026 were DeepSeek V4.1 Flash 1017 (Max) and GLM-5.3 (Max), both at an index of 32, against 59 for the overall leader, and both are far too large for a laptop. Our Artificial Analysis page explains that table. In short, the models you can run on one machine are a step down from the giants, and you pay for privacy and control with some capability.

Work out what fits

The formula

Memory for weights is the parameter count times bytes per parameter. This is our arithmetic from the GPU requirements page:

Real files run larger. The listed Llama 3.3 70B at 43 GB against the 35 GB estimate shows a gap of around 20 percent. When a model page gives a size, use it.

The cache you forget

While a model reads and writes, it keeps a key-value cache that grows with context. Ollama's FAQ says the memory needed scales with the number of parallel requests times the context length. A 14 GB model on a 16 GB card works for short chats and fails for 64K coding sessions. Add 20 to 30 percent to the listed size, then pick the card.

Ollama also sets a default context from your VRAM: 4K tokens under 24 GiB, 32K from 24 to 48 GiB and 256K at 48 GiB or more. Coding agents need more than 4K. Ollama advises 64K or higher for Claude Code, Codex and OpenCode, and 32K or more for Cline. Our Ollama models guide shows how to set it.

The special case of gpt-oss

OpenAI post-trained gpt-oss with the mixture-of-experts weights in MXFP4, about 4.25 bits per parameter, and those weights are over 90 percent of the parameters. That is why the 120B model fits one 80 GB GPU and the 20B model runs within 16 GB. See our gpt-oss page.

Tier 1: 8 GB (small laptops, older GPUs)

At this size you get chat help, summaries, commit messages and simple code snippets, not long agent runs.

Phi is the cleanest licence story here: MIT on every model we checked, per our Phi models page. Pick a coding tag for code, since the qwen2.5-coder family is built for generation, reasoning and fixing, per the Ollama library.

Tier 2: 16 GB (most modern laptops and Macs)

This is the sweet spot for a first serious local model.

If you want one answer for 16 GB, it is gpt-oss-20b. It has low, medium and high reasoning levels, works with tool calling in agent tools, and needs OpenAI's harmony response format, which serving engines handle for you. Its card puts it within 16 GB.

A caution on a 16 GB machine: with the 4K default context, an agent can fail in odd ways. Raise the context and check that the model still fits.

Tier 3: 24 GB (a single consumer GPU)

A 24 GB card, such as an RTX 4090 at 24 GB, runs the 30B class when quantised.

Tight is the word. On 24 GB, the model leaves limited room for the cache, so long agent sessions need a smaller context or a lighter model. The Qwen page lists a 24 GB GPU at 4-bit as "tight; short context only". Our recommendation from the sizes: the 30B-class coding model for code-first work, or Gemma 4 31B QAT for general use with image input.

Tier 4: 32 to 64 GB (a large Mac or a 48 GB card)

Unified memory Macs and 48 GB workstation cards open the 27B to 70B range.

Qwen3.8 27B is the one to look at first. It is dense, Apache-2.0, reads images and video, and has a 262,144 native context. Its card lists SWE-bench Pro at 61.7 and Terminal Bench 2.1 at 73.0, both vendor-reported, and it lists a different version, Terminalbench 4, at 5.6. Do not read the two against each other. See the Qwen models page and our best open source LLM for coding comparison.

Tier 5: one 80 GB GPU

An 80 GB card, such as an H100, is a rented or purchased data centre part. It makes the 100B class a single-card job.

For a team, gpt-oss-120b on one 80 GB GPU with vLLM is the cleanest route we list. Per our GPU page, you can rent one: Runpod Community lists an H100 SXM at $2.69 an hour and a Secure A100 80 GB at $1.79. Rental passes $1,200 a month at 720 hours, so price owning against renting if it runs all day. See Runpod, Lambda Cloud and Modal.

Tier 6: multi-GPU and server class

The strongest open coding models are not single-machine jobs.

KTransformers, linked from the GLM-5.3 card, is a route people use for huge mixture-of-experts models with part of the weights in system RAM. We did not test it. For most readers, these are API models. See DeepSeek models, GLM models and Nemotron models.

Choosing a runtime

Our comparison pages go deeper: Ollama vs LM Studio, llama.cpp vs Ollama and Ollama vs vLLM. A one-person setup rarely needs vLLM. A team of five sharing a model does.

Ollama cloud is worth understanding because it blurs the word "local". Models with a cloud tag run on Ollama's servers, not your machine. If privacy is the reason you want local, check that the model you pull is not a cloud one. Our Ollama page covers privacy.

First run

ollama pull gpt-oss:20b
ollama run gpt-oss:20b

Check that it is loaded and how much memory it takes:

ollama ps

For an agent, set the context first:

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

Then point your tool at the local endpoint. Our guide to a local coding agent with Ollama has exact configuration for six agents, and run an LLM locally covers the basics.

What local models are good at, and not

Good at: private chat, summarising internal notes, commit messages, small refactors, code explanation, offline use, fixed cost after you own the hardware, and experiments with no per-token bill.

Weaker at: long autonomous sessions on a large repository, hard debugging across many files, and anything where the strongest closed models lead. Makers' own figures put the best open models close to closed ones on some tests, but those models need server hardware. Compare prices on our LLM API prices page before you buy a GPU; at low volume, a cheap API such as DeepSeek V4.1 Flash at $0.15 in and $0.60 out per million tokens off-peak can cost less than the electricity and the card.

Setup cost: the weights are free and the hardware is not. Use the best LLM for coding page when you want a hosted comparison, and self-hosted AI when a team will run the model.

Checklist before you download

1. Find the model's listed size on its page.
2. Add 20 to 30 percent for the cache.
3. Check the licence for your use.
4. Check the runtime supports your card.
5. Set the context length your tool needs.
6. Run five tasks from your own work.
7. Keep the smallest model that passes.

Our take

For most people the answer is plain. Start with gpt-oss-20b on 16 GB, move to a 4-bit Gemma 4 31B or a 30B coding model on 24 GB, and use gpt-oss-120b on a single 80 GB GPU for a team. Put Ollama in front of it, raise the context, and treat the first week as a test. We have not run these models ourselves for this page; the figures are listed sizes and vendor cards, so verify on your machine.

What we could not confirm

• Speed in tokens per second on any specific machine. Our pages list sizes, not benchmarks.
• Quality of the 2-bit and 4-bit builds against full precision.
• Whether every default Ollama tag uses Q4KM. Check the tag you pull.
• GPU support lists for Ollama, which the pages we read do not give.