Artificial Analysis: Index, Speed and Price Explained

Artificial Analysis is an independent testing site that scores AI models on quality, speed, latency and price. Its headline number is the Intelligence Index, version 5.0 on 31 October 2026, a weighted average of nine evaluations. Gemini 4 Carbon (high) led it at 59 on the leaderboard we read that day, ahead of Claude Opus 5.5 at 58.

The short version

If you want the human-vote view of the same models, read our page on LMArena for how Arena works; the two answer different questions. For a quick way to judge any score, see how to read AI benchmarks.

What Artificial Analysis measures

There are two families of measures, and mixing them up is the most common mistake.

Quality. The Intelligence Index combines several evaluations into a single score. Artificial Analysis runs the evaluations itself, on its own harness for the agentic ones, rather than copying vendor claims.

Delivery. Speed, latency and price describe how a model behaves when you call it through a hosted endpoint. Artificial Analysis defines an endpoint as "a hosted instance of a model that can be accessed via an API," and notes that one model can have endpoints on several providers. That is why the site also compares providers, not only models. The performance numbers describe "the end-to-end performance experienced by customers of AI inference services and systems," not peak hardware speed.

The definitions

Artificial Analysis spells out its terms. We quote them closely because the names look alike and mean different things.

Two details deserve attention. First, the speed figures in Time per Task use the median answer tokens per second on a 10,000 input token workload, using the model's native tokenizer. A tokens-per-second number from a different tokenizer is not directly comparable. Second, the blended price assumes mostly cached input. If your workload has no prompt caching, your real cost per token sits higher than the blended figure suggests. Check the list prices on our LLM API price table.

The methodology page does not say how often provider benchmarks are run, and it describes its reasoning-token averages as coming from a diverse set of 60 prompts, some written by Artificial Analysis and some sourced from MMLU Pro, AIME 2025 and LiveCodeBench.

The Intelligence Index v5.0

The methodology page describes version 5.0 as a weighted average of nine evaluations in four categories, each worth 25 percent of the total.

Some notes on how to read that table:

• Coding is a quarter of the index, split between FrontierCode 2.0 and Terminal-Bench 4.0. A model can have a strong index and a weaker coding profile, or the reverse. The page we read for the models list did not show a separate Coding Index, so we cannot quote one.
• AA-Briefcase Elo is converted to an index contribution with a clamp of (Elo minus 500) divided by 2000, using a frozen Elo value at the time a model is added. That means the Elo-style piece does not drift when later models are added.
• Terminal-Bench 4.0 here is the public benchmark we describe on our Terminal-Bench page. The index uses its pass@1 result.
• HLE-Diamond is related to the exam we cover on the Humanity's Last Exam page. Note the name is different: the v5.0 change list says Humanity's Last Exam itself was removed from the index, while the nine-evaluation table includes the HLE-Diamond subset.
• GDP.pdf is not the same as the GDPval-AA test that earlier versions used. Our GDPval page covers the original benchmark.

Run settings

The page lists general settings. They affect how much you can trust a comparison.

• Temperature is 0 for non-reasoning models and 0.6 for reasoning models, unless the model lab recommends otherwise.
• Max output tokens is 16,384 for non-reasoning models, lowered where context or output caps require it. For reasoning models it is the maximum the model creator discloses.
• Up to 30 automatic retries on API failures. Questions that still fail are manually reviewed.
• Scoring is pass@1 generally, aggregated across repeats. Prompting is zero-shot, with no examples.
• e2b is the primary sandbox provider for agentic benchmarks.
• Repeats vary. AA-Briefcase, AutomationBench-AA, AA-Omniscience and HLE-Diamond use one run each. GDP.pdf and CritPt v2 use five. FrontierCode 2.0, Terminal-Bench 4.0 and Terminal-Bench-Science use three.
• The index is primarily text-based and English-language, though some evaluations also send images to models that accept them.

One run per evaluation on a quarter of the index means small gaps can be noise. A one-point lead between two models is not a result you should act on.

Current figures, as shown on 31 October 2026

These are the numbers the pages displayed when we read them. Artificial Analysis marks its sections "Updated" without printing a date on the pages we read, so the date here is our read date. We did not run any of these tests.

Top of the models table

The context window shown for all ten was 1M. Read three things in this table.

First, the same model appears several times at different effort settings, and the settings change the price a lot. GPT-6 Astra at max scored 57 for $6.24 a task, while at high it scored 55 for $3.24. GPT-6.1 Sol at max scored 55 for $1.27, the same score as Astra at high or xhigh for about a third to two fifths of the cost.

Second, the latency column here is first chunk time on a task, and for reasoning runs it includes thinking. Values of 118 to 729 seconds are not the sub-second latency you see on a chat model. They show how long the model works before the first output chunk. Do not compare that column with the time-to-first-token numbers below.

Third, the page shows cost per task, not a blended price per million tokens. To see what the vendor charges per token, use our model pages: Claude models, GPT models and Gemini models.

Leaders in other categories

The models page also listed these leaders on 31 October 2026.

A blended price of $0.00 is what the page displayed. We do not know whether that reflects a free endpoint, a rounding of a very small number, or a promotion, so treat it as "listed as free" and check the provider before you plan around it.

The open-weights line is worth putting next to the top of the table. The best open entries scored 32 against 59 for the leader, a gap you should weigh against price. Our pages on DeepSeek models and GLM models cover licences, prices and hardware for those two.

How to use it to pick a model

Step 1: Decide what you are optimising

Write down which of four things you care about most: quality, speed, delay before the first token, or cost. Interactive autocomplete cares about latency. A nightly batch job cares about cost. An agent that runs for an hour cares about quality and cost per task, and speed matters less.

Step 2: Filter by index, then cost per task

Start with the index to drop models that are clearly too weak. Then sort by cost per task rather than per token. A model that thinks for a long time can cost more per task than a pricier-per-token model that finishes quickly. The Sonnet 5.5 (max) row above shows the effect: an index of 54 at $10.02 a task, against GPT-6.1 Sol (max) at 55 for $1.27.

Step 3: Match the effort setting to your tool

Index scores belong to a specific effort setting. If your coding agent calls the model at medium effort, the score for max effort does not describe what you will get. Check what your agent sets. Our pages on Claude Code and Cursor describe how they pick models.

Step 4: Pick the host as a separate decision

The same model can run on several providers with different speed, latency and price. Artificial Analysis benchmarks endpoints, so use it to compare hosts for one model. Then read the host's own page for data terms, limits and plans. Our hosting pages for Groq, Together AI, Fireworks AI, OpenRouter and AWS Bedrock are a start, and AI hosting compares the kinds of provider.

Step 5: Confirm on your own work

Pull ten recent tickets or tasks from your own repository. Run two or three finalists at the settings you will use. Count tasks that pass your tests, and note the cost. This is the step no aggregate can replace.

A worked example

Suppose you run a coding agent against a large monorepo and want the lowest cost that stays near the top.

1. From the table above, shortlist models scoring 52 or more: Astra, Sol, Opus 5.5 and Sonnet 5.5.
2. Sort by cost per task. GPT-6.1 Sol (xhigh) is listed at $0.76 with an index of 52, and Sol (max) at $1.27 with 55.
3. Check Sol's per-token price on our GPT models page: $2 input and $10 output per million tokens.
4. Check whether your agent can call it at that effort level.
5. Test it against Opus 5.5 on your ten tickets.

You might still choose Opus for hard tasks. The point is that the table gave you a costed shortlist in minutes.

Limits and what to watch for

The index is a blend. A single number hides which category drove it. Two models with the same index can be strong in different places.

Benchmarks change between versions. The methodology page lists what each version added and removed. Version 5.0 replaced CritPt with CritPt v2 (Hard Subset), and removed Humanity's Last Exam, SciCode, AA-LCR and GDPval-AA from the index. Version 4.3 replaced Terminal-Bench 2.1 with Terminal-Bench 4.0. Version 4.2 removed GPQA Diamond. Version 4.1 replaced Terminal-Bench Hard with Terminal-Bench 2.1, replaced 𝜏²-Bench Telecom with 𝜏³-Banking and removed IFBench. Version 4.0 removed MMLU-Pro, LiveCodeBench and AIME 2025. An index score from an older version is not comparable to today's. Always note the version.

Some evaluations are private, and that cuts both ways. The page says AA-AnalystAgent is privately held to limit contamination, and MLCR-AA uses a private held-out set. That helps against memorisation. It also means you cannot inspect the questions.

Contamination warnings on public sets. For HLE the page notes that its authors disclose adversarial question selection based on tests with GPT-4o, and discourages direct comparison with models that were not used in that curation.

Comparability with vendor numbers. The page says some of its numbers are not directly comparable with the benchmark owner's published results because the evaluation setup differs, with Harvey LAB-AA as an example. The same caution applies when you put an Artificial Analysis figure next to a launch-post figure.

Confidence. The page notes for Code2Video that confidence intervals reflect only comparison sampling, "not variation across briefs or repeated runs." Many evaluations run once, so treat small gaps as ties.

Language and modality. The index is primarily text and English.

Independence. The page describes several evaluations as independent implementations on its own Stirrup harness. It does not state any funding sources or relationships, so we cannot comment on funding.

Snapshots. The pages we read did not carry a date. If you cite a figure, cite the day you read it.

How it compares with other leaderboards

Use them together. A model that is high on the index, high on WebDev votes and strong on test-based coding runs is a safe shortlist. A model that shines on only one of them deserves a closer look.

Our take

Artificial Analysis is the most useful single page for a price and speed shortlist, because it puts quality, cost per task and delivery speed side by side with a published method. It earns trust through stated run settings and a visible change log of the index. It loses trust when you read it as a verdict: one run per test, a text-heavy mix and a blended number are not enough to pick a coding model.

Our routine is simple. Shortlist from the index and cost per task, check the vendor price on our own pages, look at a human-vote board and a test-based benchmark, then run your own tasks. Our best open source LLM for coding page and the Claude Opus page show how we record vendor-reported scores next to prices.

What we could not confirm

• The pages we read printed no update date, so every figure here is dated by our read.
• No Coding Index appeared on the models page, and the methodology page did not define a Coding Index or Agentic Index.
• The methodology page did not say how often provider benchmarks run.
• We did not find funding or ownership details on the pages we read.
• We did not read the per-evaluation descriptions of FrontierCode 2.0 or Terminal-Bench 4.0 beyond the weights and scoring rows.
• We did not run any model.