LLM Benchmarks in 2026: Reasoning, Math, Coding & Agents

LLM benchmarks are standardized tasks and scoring procedures used to evaluate language models on abilities such as reasoning, mathematics, coding, and factual accuracy. By 2026, classic tests including MMLU, HellaSwag, GSM8K, and HumanEval provide limited separation among leading systems, so high scores are more useful for continuity and regression checks than for declaring a new frontier leader.

The most useful LLM benchmarks in 2026 depend on the task: HLE-Diamond and BBEH for harder reasoning, Terminal-Bench 4.0 for terminal agents, OSWorld 2.1 for computer use, SimpleQA Verified for factuality, and ArXivMath for research mathematics. Keep classic tests for regression checks, but compare frontier systems using current versions, transparent protocols, and independently checked, task-specific evidence.

“Outdated” does not mean useless. A saturated benchmark can still reveal regressions or differences between smaller models. A diagnostic tests a specific capability; a frontier test still leaves meaningful headroom for leading systems. These are editorial assessments as of October 2026, not permanent properties or universal claims across all setups.

October 2026 model snapshot: current releases include OpenAI’s GPT‑6 Astra and GPT‑6.1 Sol; Anthropic’s Claude Fable 5.1, Opus 5.5, and Sonnet 5.5; and Google’s Gemini 3.8 Flash.

Access differences: Claude Mythos 5.1 is offered through trusted access programs, while Gemini 4 Argon, announced September 30, is initially rolling out to trusted cyber defenders. Compare the exact model version and access conditions alongside its scores. Our earlier comparison of Kimi K2, DeepSeek-R1, Qwen3, and GLM-4.5 remains a worked example of comparing open-weight models, rather than an October release list.

HellaSwag tests whether a model can select a plausible continuation of an everyday scenario. In 2026, use it as a commonsense diagnostic and historical reference rather than a decisive frontier ranking.

WinoGrande tests whether a model can resolve an ambiguous reference using commonsense context. In 2026, it is most useful for tracking this narrow skill alongside harder reasoning evaluations.

PIQA tests practical knowledge of how everyday objects and physical actions work. Use it in 2026 to check basic physical commonsense, not to establish broad reasoning or real-world competence.

SocialIQA tests reasoning about people’s intentions, reactions, and likely social consequences. In 2026, its short multiple-choice scenarios offer a targeted diagnostic rather than a measure of social judgment in deployment.

OpenBookQA tests whether a model can combine elementary science facts with commonsense knowledge. In 2026, it remains useful for reproducible diagnostics but should be paired with more demanding science evaluations.

ARC tests grade-school science knowledge and reasoning through multiple-choice questions. In 2026, use its Easy and Challenge splits for continuity, and distinguish it from the unrelated ARC-AGI puzzle benchmarks.

CommonsenseQA tests how models reason about everyday concepts and their relationships. In 2026, treat its static multiple-choice questions as a focused diagnostic rather than sufficient evidence of frontier reasoning.

MMLU tests academic and professional knowledge across a broad set of subjects. In 2026, retain it for historical comparison while checking harder successors and domain-specific tasks for meaningful differences.

MMLU-Pro tests academic knowledge and reasoning with harder questions and more answer choices than MMLU. It can complement a 2026 evaluation suite, provided comparisons account for prompting and possible exposure to the static dataset.

GPQA Diamond tests expert-level scientific reasoning in biology, chemistry, and physics. In 2026, it is useful for probing difficult science questions, but its small size and evaluation conditions matter when interpreting score differences.

Humanity’s Last Exam tests advanced academic knowledge and reasoning with text-only and image-based questions. In 2026, use it to probe difficult problems while comparing like-for-like tool access and modality settings.

October 2026 update — HLE-Diamond: The HLE team released this 1,000-question refined set on September 22, 2026, split evenly between reasoning and knowledge. Treat it as a newer complement and migration path as the original HLE approaches its noise ceiling—not as another name for HLE’s 2,500-question public set. Always separate tool-enabled from closed-book results. Official announcement.

BIG-Bench Hard tests a varied set of algorithmic, symbolic, and multi-step reasoning skills. In 2026, its main value is diagnostic continuity; harder complements are needed when scores no longer separate the models being compared.

BBEH tests a broad mix of difficult reasoning skills through tasks such as logical deduction, tracking information and interpreting rules. It is the explicit harder successor to BIG-Bench Hard: each BBH task is replaced by a new task targeting a similar ability at substantially higher difficulty.

Size and format: 23 tasks, with 4,520 examples in the full benchmark and 460 in BBEH Mini; answer formats vary by task.

How to use it: Prefer BBEH when BBH no longer separates strong models. Report whether a result uses the full or Mini set and whether it is the official harmonic-mean score or a micro-average—those scores are not interchangeable. Official benchmark; BBEH paper.

ARC-AGI-2 tests whether a system can infer unfamiliar visual rules from examples and apply them to new grids. In 2026, interpret its results alongside the evaluation split and inference cost rather than as a general intelligence score.

Competition math is no longer a reliable frontier separator by itself. In May 2026, MathArena reported near-ceiling AIME and HMMT performance and recommended these tests mainly for smaller models and method comparisons. That was not a claim that all mathematics—or every final-answer benchmark—was solved. MathArena is now an evaluation platform spanning competitions, research questions, reliability, and formal proofs. MathArena’s May 2026 paper.

Keep three questions separate: can the model obtain the answer, justify it in a natural-language proof, and produce a machine-checked proof of the intended statement? “Research-level” describes the source and difficulty, not necessarily proof grading. Even FrontierMath’s hardest closed-answer tier has reached its ceiling; newer research and formal-verification tasks are the more informative complements, not a blanket guarantee of difficulty.

GSM8K tests multi-step arithmetic reasoning through grade-school word problems. In 2026, it is a useful regression check, but stronger models need harder and fresher math evaluations to reveal meaningful weaknesses.

MATH tests competition-style mathematical problem solving across several subjects and difficulty levels. In 2026, use it for continuity alongside fresh problems and checks of solution quality, especially where training exposure is uncertain.

MGSM tests whether mathematical reasoning transfers across languages using translated grade-school problems. In 2026, report language-level results and treat this small shared problem set as a starting point for multilingual evaluation.

MathArena’s AIME 2026 comprises 30 integer-answer questions; HMMT February 2026 uses 33 selected final-answer questions, excluding unsuitable proof/construction tasks. Specify the year and subset. These remain useful regression tests, but their near-ceiling results should not headline comparisons of the strongest models. Dataset definitions.

This is proof assessment, not answer matching. GPT-5.4 scored 95.24% of normalized rubric points across six problems and four runs per problem. MathArena used an LLM jury plus human review; other evaluated models scored substantially lower. A near-perfect result on this small contest does not establish general proof mastery. Grading and results; scoreboard.

Epoch’s June 2026 v2 contains 295 Tiers 1–3 problems and 43 Tier 4 problems; the private scored sets contain 285 and 41 respectively. Models submit computationally checkable answers with Python access—not formally verified proofs. The correction changed many questions, so do not mix v1 and v2 scores. Epoch called Tier 4 saturated in September; its October results support treating the closed-answer tiers as ceiling-limited rather than automatically frontier-discriminating. Version and protocol; Epoch data; saturation assessment.

MathArena turns recent research-paper results into self-contained, final-answer questions. Its September 2026 cohort has 53 problems; fresh cohorts reduce, but do not eliminate, contamination risk. Report the cohort, not just “ArXivMath.” The September cohort already has high scores and warnings for models released after the tasks, so freshness alone is not evidence of uncontaminated evaluation. Construction; September results.

A distinct, refreshable research-math pipeline, not another name for ArXivMath. The released dataset has 1,286 questions: 633 Math.arXiv, 111 CS.arXiv, and 542 Math.StackExchange. It grades numerical/symbolic answer equivalence using an LLM judge, not the validity of a full proof. Its published model results are a 2025 snapshot; do not treat them as a current frontier ranking. Paper and evaluation; released data.

Research statements require proofs or disproofs accepted in a fixed Lean environment, with checks that preserve the intended statement and permitted axioms. September 2026 has 53 conjectures in Lean 4.34.1. The checked results leave meaningful headroom, but newer-model rows carry release-after-competition warnings. Pin the cohort, Lean/Mathlib version, tools, and budget. Cohort specifications; verification safeguards; results.

A separate open-problem benchmark: 68 Lean conjectures covering 65 numbered problems, all open when selected in August 2026. Each gets one agent attempt capped at $300 and 72 hours; Comparator checks a proof or disproof. The checked results remain far from ceiling. This tests research plus formalization, not all branches of mathematics. Protocol; scores.

Scores checked October 25, 2026. Do not equate answer accuracy, partial-credit proof points, and formal proof success. MathArena’s newer-model warning identifies a release-date risk, not demonstrated contamination.

HumanEval tests whether generated Python functions pass executable tests. In 2026, use it as a small, familiar coding baseline and pair it with fresher problems, repository-level work, and private tests.

MBPP tests entry-level Python programming from short natural-language descriptions. In 2026, it is useful for basic code-generation diagnostics, but limited test coverage makes stronger tests and more realistic tasks important complements.

LiveCodeBench tests coding capabilities on programming-contest problems collected over time. For 2026 comparisons, its release dates make it useful for studying fresher tasks, but the chosen time window and evaluation scenario must be explicit.

For example, release v6 contains 1,055 problems from May 2023 through April 2025. It is a versioned historical window, not a promise of unseen tasks for models released in 2026. Versioned dataset and evaluation code.

Agent evaluations ask a model to act over multiple steps: inspect a repository, use a terminal, search the web, operate applications, or help a customer through a transaction. They test planning, tool use, recovery, and persistence. Read every result as a system score: the model, agent harness, tools, budget, benchmark version, and grading rule all matter.

SWE-bench Verified tests repository repair. It remains a useful historical reference, but it is no longer a strong standalone frontier test.

SWE-Bench Pro extends repository repair to larger, multi-file engineering tasks. Specify the release: the original public split and the cleaned V2 are different evaluations.

Terminal-Bench tests whether an agent can finish difficult work through a command line, including coding, scientific computing, systems work, and operational tasks.

BrowseComp measures persistent web research: finding an obscure answer by connecting clues across sources.

OSWorld evaluates computer use in real desktop and web applications, including workflows that move between them.

The τ-bench family tests whether an agent can help a simulated customer while following policies and using transactional tools.

Natural Questions tests answering real search queries using evidence from Wikipedia pages. In 2026, it remains useful for retrieval and reading-comprehension research, without by itself testing current web browsing or research agents.

TriviaQA tests question answering with independently collected evidence documents. In 2026, use it as a retrieval and reading-comprehension baseline, complemented by fresh questions and separate factuality checks.

DROP tests reading comprehension that requires operations such as counting, addition, sorting, and date comparison over a passage. In 2026, it remains a targeted diagnostic of passage-grounded reasoning rather than a broad math benchmark.

MMMU and MMMU-Pro test academic reasoning that combines images and text across many disciplines. In 2026, use them within a broader visual evaluation suite and distinguish their settings from audio, video, or real-world vision tasks.

The released MMMU dataset contains 900 validation and 10,500 test questions; the test answers became public in February 2026. MMMU-Pro has 1,730 underlying questions per setting, not 5,190 independent questions across its three configurations. MMMU release notes.

TruthfulQA tests whether models resist common misconceptions instead of repeating plausible falsehoods. In 2026, use it as a targeted truthfulness diagnostic alongside broader factuality and domain-specific verification tests.

Specify the TruthfulQA variant: generation, original MC1/MC2, or the newer randomized two-option evaluation. The original 817-question count does not identify every revised release. Current dataset and evaluation instructions.

SimpleQA Verified tests whether a model can answer short factual questions from its own stored knowledge, without looking up the answer. It belongs in a factuality section, not a general-reasoning leaderboard.

Size and format: 1,000 short-answer prompts, refined from OpenAI’s 4,326-question SimpleQA through deduplication, topic balancing and source reconciliation.

How to use it: Responses are graded correct, incorrect or not attempted. The authors report an F1-style combination of overall accuracy and accuracy among attempted answers; other evaluators may report simple accuracy. Keep that distinction visible, along with the judge and abstention instructions. It is not a universal hallucination rate: long-form, retrieved and tool-assisted answers need separate tests. Paper and methodology.

A benchmark score describes a model running inside a particular evaluation setup. The same model can produce different results when its prompts, tools, time limits, or agent software change. Record the setup alongside the number, rather than comparing percentages copied from unrelated launch posts.

For deployment decisions, also track latency, cost per successful task, reliability, and security. Capability tests do not replace adversarial and security evaluations or LLM observability in production.

Same model, different agent setup: Scale’s SWE-Bench Pro V2 harness study reports Inkling at 84.1% with Pi, 89.9% with mini-swe-agent, and 91.6% with OpenCode on the same 642 tasks. GLM-5.3’s scores vary much less, from 95.6% to 96.6%. Harness effects are real, but their size depends on the model. These are study results: one counted attempt per task under the paper’s retry rule, graded in the agent’s container, rather than the main leaderboard’s clean-container re-grading protocol. Source: Table 3 and Appendix B.

A second example comes from OSWorld’s June 2026 release: Claude Opus 4.7 at max effort scored 13.9% complete-task success with Standard tools and 18.2% with Batched tools. Both entries use release v2026.06.24. This illustrates an action-interface effect without mixing benchmark versions. Source: official OSWorld leaderboard.

For “LLM benchmark scores 2026” comparisons, use the maintained sources below rather than a fixed ranking in this article. Their scores answer different questions, and their datasets and methods can change.

For reproducible implementations, use the LM Evaluation Harness; Stanford HELM provides standardized evaluation across scenarios and metrics. Neither makes results comparable unless the actual configurations match.

HellaSwag is a four-choice commonsense benchmark: a model picks the most plausible continuation of an everyday situation. It is useful for historical comparisons and regression tests, but its mature, public question set gives limited separation among frontier models in 2026.

MMLU tests academic and professional knowledge across 57 subjects using four-choice questions. Original MMLU is a legacy comparison point in 2026. MMLU-Pro adds harder, more reasoning-focused questions and more answer choices, but it also needs version and contamination checks.

MMLU resembles a broad academic exam, covering subjects from mathematics to law. HellaSwag asks what plausibly happens next in an everyday scenario. They test different skills, so a higher score on one does not imply stronger performance on the other.

Scores can be distorted by training-data contamination, saturation, flawed questions, unreliable grading, or an evaluation setup unlike real work. Prompts, tools, retries, and compute budgets also affect results. Use multiple tests and production evidence, rather than treating one benchmark as a complete measure of intelligence.

Most capability benchmarks primarily grade correctness or task success. Latency and cost must usually be measured alongside that score: report wall-clock time, token usage, tool costs, hardware or API provider, and cost per completed task. Platforms such as Artificial Analysis and Vals AI report some of these operational metrics.

It depends on the mathematics and model level. GSM8K checks basic arithmetic; AIME and HMMT test competition answers; USAMO evaluates written proofs. Research-derived tasks and Lean benchmarks probe different abilities. Specify the exam year, dataset version, grading method, tools, and attempt budget before comparing results.

We post helpful lists and bite-sized explanations daily on our X (Twitter). Let’s connect!