gpt-oss: OpenAI's Open Models, Sizes, Hardware, Hosts

gpt-oss is OpenAI's open-weight pair, released on 4 August 2025 under Apache-2.0. gpt-oss-120b (117B parameters, 5.1B active) fits a single 80 GB GPU; gpt-oss-20b (21B, 3.6B active) runs within 16 GB. Both take text in and out with a 131,072-token context, three reasoning levels and tool use. Checked 31 October 2026.

The two models

All facts are from the two Hugging Face cards, the 120b config file and the Hugging Face API, read on 31 October 2026.

The Hub repositories are larger than the memory figures because they hold more than one format: the Hugging Face API lists about 196 GB of files for the 120b repository and 41 GB for 20b. The card's 80 GB and 16 GB figures describe running the MXFP4 build, which is what matters for hardware planning.

What OpenAI says about them

The card calls the pair "designed for powerful reasoning, agentic tasks, and versatile developer use cases", with the 120b aimed at production, general purpose and high reasoning use, and 20b at lower latency and local or specialised use. Its listed highlights:

• Permissive Apache-2.0. No copyleft, no patent risk worry, and commercial use allowed.
• Configurable reasoning. Set low, medium or high in the system prompt, for example "Reasoning: high".
• Full chain of thought. You see the reasoning, which helps debugging. OpenAI says it is not intended to be shown to end users.
• Fine-tuning. The 120b can be fine-tuned on a single H100 node, and the 20b on consumer hardware, per the card.
• Agentic tools. Native function calling, web browsing, Python execution and structured outputs.
• MXFP4. The MoE weights were post-trained in MXFP4, and all evaluations used that quantisation.

We found no benchmark table in the text of either card: OpenAI points to the arXiv model card for evaluations. We have not extracted scores from it:.

The harmony format

The catch with gpt-oss is the response format. The card says both models were trained on OpenAI's harmony format and "should only be used with the harmony response format as it will not work correctly otherwise". In practice:

• If you use the Hugging Face Transformers chat template, harmony is applied for you.
• If you call model.generate directly, you must apply it yourself or use the openai-harmony package.
• Serving engines such as vLLM, and apps such as Ollama and LM Studio, handle it for you.

If you build your own inference loop, this is where things break. Symptoms are odd output, missing tool calls or reasoning leaking into answers. The fix is never prompt engineering: use the harmony tooling.

Running it

The card documents five routes.

The vLLM instruction on the card pins a 2025 build (vllm 0.10.1 with a gpt-oss tag). Current vLLM releases support the model natively, but check the current docs rather than copying that pin:.

For a comparison of the serving tools, see Ollama vs vLLM and Ollama vs LM Studio.

Which hardware

The 20b's 16 GB claim makes it the cheapest route into a capable local reasoning model, and it explains why it became the default in so many local tools. The 120b is the pick when you have a data-centre GPU or rent one from Runpod, Modal or Lambda.

Hosted prices

If you do not want to run it, many hosts sell gpt-oss by the token. Two we have compared on this site list gpt-oss-120b at these rates (checked 31 October 2026):

Both speed figures are vendor claims. See Groq vs Cerebras for the full comparison. For reference, OpenAI's own closed small model, GPT-6 Luna, is priced at $0.10 input and $0.50 output per million tokens with a 1.05M-token context window, so gpt-oss-120b is not the cheapest OpenAI-lineage token any more; its reasons to exist are the licence and the control. Other hosts, including Hugging Face Inference, Cloudflare Workers AI and AWS Bedrock, have carried gpt-oss; confirm each one's current catalogue and price:.

Using it with coding agents

gpt-oss supports function calling, so any agent that talks to an OpenAI-compatible endpoint can drive it when served by vLLM, Ollama or a hosted provider. Ollama's gpt-oss tags are on the card. OpenAI's own Codex CLI is documented to work with local open-source models through Ollama, but we did not confirm the current flag: . For other agents see OpenCode, Cline and Aider.

A caution about coding quality. gpt-oss is a general reasoning model from 2025. Newer open models publish agentic coding numbers that the gpt-oss cards do not, and most compare themselves to far newer systems. See best open source LLM for coding for that comparison. gpt-oss's case is the small footprint: we know of nothing else in our lineup that combines Apache-2.0, a 5B active parameter count and a documented single-GPU fit at 117B total.

Reasoning levels in practice

The three levels are a speed and cost dial, set in the system prompt. Low gives fast answers for general dialogue, medium balances speed and detail, and high gives deep and detailed analysis, in OpenAI's words. Because the reasoning is generated text, a higher level means more output tokens, which means more latency and, on a hosted API, a larger bill on the output side that is already the dearer one ($0.60 against $0.15 on Groq).

A sensible default for agent use is to start at medium and move individual steps up or down. Planning and debugging steps benefit from high. Simple formatting, summarising and classification steps rarely need more than low. We have no measured token counts per level to quote, so log them in your own workload before you set a policy. If you run the model yourself, tokens cost GPU time rather than money, but the same logic applies: shorter reasoning frees the GPU for more requests.

Tool use, as the card describes it

The card names three built-in capabilities: web browsing with built-in browsing tools, function calling with defined schemas, and agentic operations such as browser tasks. It also lists Python code execution and structured outputs in its highlights. The reference tools live in OpenAI's gpt-oss GitHub repository, with browser and Python implementations you can run alongside the model.

Two practical notes. First, function calling goes through the harmony format, so your serving engine must translate your tool schema into harmony and parse the model's tool call back out. vLLM and Ollama do; a home-grown server may not. Second, the browsing and Python tools in the repository are reference implementations, and you are responsible for sandboxing the Python one. Do not run model-written code on a machine that holds secrets.

When running it yourself beats an API

Self-hosting only pays when volume is high enough. A rough break-even uses two figures from this site: Runpod's community H100 SXM at $2.69 an hour (see Runpod) and Groq's gpt-oss-120b rate. Assume a workload of 75 percent input tokens and 25 percent output. The blended Groq price is then $0.2625 per million tokens (0.75 times $0.15 plus 0.25 times $0.60). A $2.69 hour buys the same value at Groq's price as about 10.2 million tokens, which is roughly 2,850 tokens per second sustained for the whole hour.

That is our arithmetic, not a benchmark. It tells you the shape of the decision: unless your own server will be busy most of every hour with that much traffic, the hosted per-token rate wins on cost. The reasons to self-host are then control, privacy and predictable behaviour, not savings. Our cloud vs self-hosted AI page walks through the rest of the costs, including the engineer who keeps the server running.

Compared with OpenAI's closed models

OpenAI sells a separate closed lineup: GPT-6 Astra at $10 input and $50 output per million tokens, Sol 6.1 at $2 and $10, Terra 6.1 at $1 and $5, and Luna at $0.10 and $0.50, all with a 1.05M-token context and 128K maximum output (see our GPT models page and OpenAI API pricing). gpt-oss differs on five points.

If a task needs images, a million tokens or the strongest OpenAI reasoning, gpt-oss is the wrong tool. If it needs a model you can pin for years, audit and run behind your own firewall, it is the right one.

A deployment checklist

1. Pick the size by memory first: 16 GB for 20b, 80 GB for 120b.
2. Use an engine that applies the harmony format, and verify with a tool-calling test.
3. Set the reasoning level per request, not globally.
4. Cap output tokens, since high reasoning can run long.
5. Hide the chain of thought from end users.
6. Pin the repository revision in your deployment so a later upload cannot change your behaviour. The cards were last modified on 26 August 2025.
7. Re-evaluate against newer open models every quarter; this field moves quickly.

Fine-tuning

OpenAI says the 120b can be fine-tuned on a single H100 node and the 20b on consumer hardware, and that both can be fine-tuned for specialised use. Because the licence is Apache-2.0, your tuned model is yours to ship, with Apache's notice requirements. That is a real difference from licences that name derivatives or require attribution. Train in the harmony format or your tuned model will drift from how the base was trained. Keep a held-out evaluation set, since a model with visible chain of thought is easy to over-tune on reasoning style.

Limits and gotchas

• Age. The weights date from August 2025. They are widely supported, but newer open models report higher agentic scores on their own cards.
• Text only. No image input. Gemma 4 and Qwen3.8-27B read images; see Gemma and Qwen.
• Context. 131,072 tokens is short next to the 256K to 1M windows of most newer open models on this site.
• Format lock-in. The harmony requirement trips up custom code.
• Chain of thought exposure. The reasoning is not meant for end users; filter it in your UI.

Who should pick gpt-oss

Our take

gpt-oss is the dependable small-footprint option. It is not the highest-scoring open model, and OpenAI's own cards do not claim it is, but its Apache-2.0 licence, 80 GB and 16 GB fit, and wide tooling support still make it the easiest first model to put on your own hardware. If you already run it, there is no urgency to move. If you are choosing today for coding, compare it against Qwen3.8-27B, and for the full list see open source LLM and GPT models.

Product names are used descriptively. We are not affiliated with OpenAI.