Groq

GroqCloud serves open-weight models on its own inference hardware, built for speed. On 2026-10-31 gpt-oss-120b costs $0.15 in and $0.60 out per million tokens at about 500 tokens a second. Groq's docs say it does not retain inference data by default, and any retained data sits in US cloud storage.

Facts at a glance

Prices checked 2026-10-31. Groq is a host, not a model maker, so the models are other labs' weights. See gpt-oss for the OpenAI weights it serves most.

Models and prices

Groq splits its catalogue into Production models, meant for production use, and Preview models, which are for evaluation and may be discontinued on short notice. This matters for planning. A preview model is not something to build a product on.

Whisper speech-to-text models are also in the production list, and the catalogue lists text-to-speech and prompt-guard safety models. The speeds are vendor figures, not independent measurements. They are the reason people pick Groq, and they deserve a test on your own prompts.

Qwen3.8-27B on Groq lists a context of 131,072 tokens with 16,384 maximum output. For gpt-oss-120b the open model is Apache-2.0 with a 131,072-token context in its own card, and Groq's model table lists gpt-oss-120b at 131,072 tokens of context and 65,536 maximum completion tokens.

Rate limits and plans

Groq's rate-limit page gives limits per model in a table with these columns: requests per minute (RPM), requests per day (RPD), tokens per minute (TPM) and tokens per day (TPD). Some organizations also get separate input and output token limits, called ITPM and OTPM, and the docs say you can hover over your TPM value on the Limits page to see an "X in / Y out" breakdown.

The page says the limits shown are the base limits for the Developer plan, and that higher limits are available for select workloads and enterprise use. Upgrading to Developer unlocks higher limits, Batch and Flex processing, and more. Some limits in the table are small for the specialist models. For example, the Orpheus speech models are listed at 10 requests per minute and 100 per day, with 1.2K tokens per minute and 3.6K per day on the base table. A request counts against both RPM and TPM. The docs give the example of 50 requests of 100 tokens each in a minute, which hits a 50 RPM limit even though only 5,000 tokens were sent.

The docs describe three service tiers: Performance, Flex and Batch. Batch is asynchronous at a 50 percent discount against synchronous pricing, with a completion window of 24 hours to 7 days. Flex is priced the same as on-demand, offers 10 times the on-demand rate limits to paid customers, and fails fast with status 498 when capacity runs out. Spend limits are a console feature, so a runaway agent can be capped there. Free-tier daily limits for the main models are in the rate limit page, and we did not copy them because they change.

Data retention and location

Groq's Your Data page is short and clear.

• Usage metadata is always retained, to measure service activity and performance. It does not contain customer inputs or outputs.
• Customer data is not retained by default for inference requests.
• Customer data is retained in two cases only. If you use a feature that needs retention to function, such as batch jobs, fine-tuning and LoRAs. And if it is needed to protect platform reliability, for example to troubleshoot system failures or investigate abuse.
• Controls. You manage these settings in the Data Controls area of the console.
• Location. All customer data that is retained is held in Google Cloud Platform buckets in the United States. Groq says it keeps strict access controls and publishes its security standards in a Trust Center. Where applicable, customers can rely on standard contractual clauses for transfers between regions.

Two gaps are worth noting. Where data is kept for reliability or abuse checks, the page says logs are retained for up to 30 days unless the law requires longer, and customers can enable Zero Data Retention in Data Controls, which also switches off batch processing and fine-tuning. The page does not say whether Groq trains on customer data. For a host that serves other labs' open weights, the default of not retaining data is the more important statement, but a procurement form will want the training answer in writing from Groq.

A related point is that Groq does not serve every model everywhere. The inference runs on Groq's hardware, and the data location statement is about retained data, not about where inference runs. If you need a stated processing region, ask Groq's sales team.

A worked monthly cost

Take a job that reads 100 million input tokens and writes 10 million output tokens in a month. The arithmetic is ours, from the list prices above, and it ignores any caching discount.

The last column is only a what-if for a batch-style tier that halves the price. We do not state Groq's actual batch discount because we could not confirm it. The pattern holds regardless: the two OpenAI models cost a fraction of the preview Qwen model, so the first question is whether the smaller model passes your task test. If gpt-oss-20b does, the price falls by half again. Input-heavy workloads, such as reading a repository, lean on the input price, so the $0.075 against $0.15 gap is where the saving sits.

What the models are, and what that means for you

Groq does not make the models. gpt-oss-120b and gpt-oss-20b are OpenAI's open-weight release under Apache-2.0. OpenAI's model card says the 120b fits a single 80 GB GPU, because the mixture-of-experts weights are post-trained in MXFP4, and the 20b runs within 16 GB. The card also says both models were trained on OpenAI's harmony response format and should only be used with it. A host handles that for you, but if you move the same prompts to a self-hosted copy, your server must apply the harmony format or answers degrade. For the full model entry, see gpt-oss.

Three consequences follow from Groq being a host.

1. Quality is the lab's, speed is Groq's. If a model is weak at your task on Groq, it will be weak on any host. If it is slow elsewhere, Groq may fix that.
2. Model churn is real. Preview models can be dropped on short notice, and Groq's catalogue changes. Pin the model id in config and watch Groq's changelog.
3. Precision is the host's choice. Hosts sometimes shrink models to run faster. We did not find a Groq statement on quantisation in the pages we read.

Moving to Groq from another provider

Because the API is OpenAI-shaped, the code change is small, and the testing is not.

1. Change three values. Base URL to https://api.groq.com/openai/v1, the key, and the model id.
2. Check tool calling. Groq documents a tool use overview, built-in tools and remote MCP connectors. Behaviour for a given model depends on the model, so run your tool schemas through it.
3. Check structured output. Groq lists structured outputs as a core feature. Test your schema on the exact model.
4. Right-size the model. The big saving comes from picking the smallest model that passes, not from the vendor swap.
5. Plan limits. TPM and RPM are per model. A test that passes on one request will not tell you what a hundred parallel agents do.
6. Review retention. If you adopt batch, fine-tuning or LoRAs, data is retained for those features, and the default no-retention statement no longer covers them.

Moving away is the reverse. Since most agents speak the OpenAI format, a move to Together AI or Fireworks AI is also a base-URL change, and each host names its models differently, so check the id.

Common errors and what they mean

The error codes are listed in a Groq docs page, which is the place to confirm exact status behaviour.

Which agents can use it, and how

Groq publishes a guide for coding agents. OpenCode has native Groq support. Inside the OpenCode terminal interface you connect and then choose a model:

/connect      # search for Groq, paste your key from console.groq.com/keys
/models       # pick a Groq model, for example openai/gpt-oss-120b

OpenCode stores the key in ~/.local/share/opencode/auth.json. Groq's docs list openai/gpt-oss-120b as the most popular choice for complex coding tasks. The docs navigation also lists guides for Factory Droid, Kilo Code, Roo Code and Cline. See OpenCode and Cline.

For any other OpenAI-compatible agent or SDK, the base URL is https://api.groq.com/openai/v1:

curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-oss-120b","messages":[{"role":"user","content":"Explain this stack trace."}]}'

Claude Code and Codex CLI are built for their own vendors' models, so a Groq route works only through an adapter or gateway, which we have not tested. OpenRouter lists Groq-served models through its routing.

What speed is worth

Speed matters when a person or an agent waits for the answer. A coding agent that makes forty model calls spends most of its wall-clock time waiting. By the vendor figures, a 1,000-token reply takes about two seconds at 500 tokens a second. Forty such calls come to about 80 seconds of generation. At 3,000 tokens a second, the figure Cerebras lists for the same model, it is about 13 seconds. For overnight batch work, speed matters little and the price per token matters most, which is where Groq's list price on gpt-oss-120b is lower than Cerebras's. Our head-to-head is on Groq vs Cerebras.

Alternatives

A pre-launch checklist

1. Choose Production models only for anything customer-facing.
2. Check the Limits page for your plan and note TPM and RPM for your models.
3. Set a spend limit in the console.
4. Turn on or review the Data Controls settings.
5. Decide whether batch features, which retain data, are acceptable for your data.
6. Ask for the training and retention-period statements in writing if a review needs them.
7. Test speed on your own prompts, since the figures are vendor-reported.

Our take

Groq is a good fit when latency is the product: a coding agent, a voice assistant, anything with many sequential calls. The no-retention default is the best-stated of the open-model hosts we read, and the US storage statement is simple. The costs are the narrow model list compared with the larger hosts, preview models that can disappear, and a catalogue of other labs' weights, which means model quality is whatever those labs shipped. For open-weight work at low price per token with data you would rather not have stored, it is a strong first choice.

We have not run tests for this page. It reflects Groq's published documentation on 2026-10-31. Groq does not pay for placement here. Affiliate links on this site, where used, go through /go/ with rel="sponsored".

Related pages

Groq vs Cerebras, gpt-oss, open source LLM, LLM API prices, AI hosting.