Benchmark

Humanity's Last Exam Benchmark

What is Humanity's Last Exam?

Humanity's Last Exam (HLE) is a multi-modal academic benchmark with 2,500 questions across mathematics, humanities, and natural sciences, designed to test LLM…

Released
2025-01-24
Modality
multimodal
Evaluates
math, reasoning, vision
Openness
restricted
Importer
LLM Stats
Reported scores
99

Source provenance

Humanity's Last Exam paper, code and dataset

Humanity's Last Exam results and reported scores

LLM Stats

ModelOrganizationReported valueReportedEvidence
Claude Fable 5Anthropic0.6452026-06-09evidence
Claude Mythos PreviewAnthropic0.6472026-04-07evidence
Claude Opus 4.6Anthropic0.5312026-02-05evidence
Claude Opus 4.7Anthropic0.5472026-04-16evidence
Claude Opus 4.8Anthropic0.5792026-05-28evidence
Claude Opus 5Anthropic0.6472026-07-24evidence
Claude Sonnet 4.6Anthropic0.492026-02-17evidence
Claude Sonnet 5Anthropic0.5742026-06-30evidence
DeepSeek-R1-0528DeepSeek0.1772025-05-28evidence
DeepSeek-V3.1DeepSeek0.1592025-01-10evidence
DeepSeek-V3.2DeepSeek0.4082025-12-01evidence
DeepSeek-V3.2 (Thinking)DeepSeek0.2512025-12-01evidence
DeepSeek-V3.2-ExpDeepSeek0.1982025-09-29evidence
DeepSeek-V3.2-SpecialeDeepSeek0.3062025-12-01evidence
DeepSeek-V4-Flash-0423DeepSeek0.4032026-04-23evidence
DeepSeek-V4-Flash-MaxDeepSeek0.4512026-04-23evidence
DeepSeek-V4-Pro-0813DeepSeek0.62026-08-13evidence
DeepSeek-V4-Pro-MaxDeepSeek0.4822026-04-23evidence
DiffusionGemma 26B-A4BGoogle0.1192026-06-10evidence
ERNIE 5.0Baidu0.392026-01-22evidence
Gemini 2.5 FlashGoogle0.112025-05-20evidence
Gemini 2.5 Flash-LiteGoogle0.0512025-06-17evidence
Gemini 2.5 ProGoogle0.1782025-05-20evidence
Gemini 2.5 Pro Preview 06-05Google0.2162025-06-05evidence
Gemini 3 FlashGoogle0.4352025-12-17evidence
Gemini 3 ProGoogle0.4582025-11-18evidence
Gemini 3.1 Flash-LiteGoogle0.162026-03-03evidence
Gemini 3.1 ProGoogle0.5142026-02-19evidence
Gemini 3.5 FlashGoogle0.4022026-05-19evidence
Gemma 4 12BGoogle0.0522026-05-23evidence
Gemma 4 26B-A4BGoogle0.1722026-04-02evidence
Gemma 4 31BGoogle0.2652026-04-02evidence
GLM-4.5Z.ai0.1442025-07-28evidence
GLM-4.5-AirZ.ai0.1062025-07-28evidence
GLM-4.6Z.ai0.1722025-09-30evidence
GLM-4.7Z.ai0.4282025-12-22evidence
GLM-4.7-FlashZ.ai0.1442026-01-19evidence
GLM-5.1Z.ai0.5232026-04-07evidence
GLM-5.2Z.ai0.5472026-06-16evidence
GLM-5.3Z.ai0.6252026-08-14evidence
GPT OSS 120BOpenAI0.1492025-08-05evidence
GPT OSS 20BOpenAI0.1092025-08-05evidence
GPT-4.1OpenAI0.0542025-04-14evidence
GPT-4.1 miniOpenAI0.0372025-04-14evidence
GPT-4oOpenAI0.0532024-08-06evidence
GPT-5OpenAI0.2482025-08-07evidence
GPT-5 miniOpenAI0.1672025-08-07evidence
GPT-5 nanoOpenAI0.0872025-08-07evidence
GPT-5.2OpenAI0.3452025-12-11evidence
GPT-5.2 ProOpenAI0.3662025-12-11evidence
GPT-5.4OpenAI0.3982026-03-05evidence
GPT-5.4 miniOpenAI0.2822026-03-17evidence
GPT-5.4 nanoOpenAI0.2432026-03-17evidence
GPT-5.5OpenAI0.5222026-04-23evidence
GPT-5.5 ProOpenAI0.5722026-04-23evidence
Grok 4 FastxAI0.22025-08-28evidence
Grok-4xAI0.42025-07-09evidence
Grok-4 HeavyxAI0.5072025-07-09evidence
Inkling-SmallThinking Machines Lab0.3162026-07-30evidence
Kimi K2 InstructMoonshot AI0.0472025-07-11evidence
Kimi K2-Instruct-0905Moonshot AI0.0472025-09-05evidence
Kimi K2-Thinking-0905Moonshot AI0.512025-09-05evidence
Kimi K2.5Moonshot AI0.5022026-01-27evidence
Kimi K2.6Moonshot AI0.3642026-04-20evidence
Kimi K3Moonshot AI0.562026-07-16evidence
LongCat-Flash-Thinking-2601Meituan0.2522026-01-14evidence
Magistral MediumMistral0.092025-06-10evidence
MAI-Code-1-FlashMicrosoft0.182026-06-02evidence
MiMo-V2-FlashXiaomi0.2212025-12-16evidence
MiMo-V2.5-ProXiaomi0.342026-04-27evidence
MiniMax M1 40KMiniMax0.0722025-06-16evidence
MiniMax M1 80KMiniMax0.0842025-06-16evidence
MiniMax M2MiniMax0.1252025-10-27evidence
MiniMax M2.1MiniMax0.222025-12-23evidence
Muse Glimmer-30BMeta0.222026-08-10evidence
Muse SparkMeta0.5842026-04-08evidence
Muse Spark 1.1Meta0.6212026-07-09evidence
Nemotron 3 Nano (30B A3B)NVIDIA0.1552025-12-15evidence
Nemotron 3 Super (120B A12B)NVIDIA0.22822026-03-11evidence
Nemotron 3 Ultra (550B A55B)NVIDIA0.3742026-06-04evidence
Nemotron 3.5 Lightning (30B A3B)NVIDIA0.11722026-08-11evidence
o3OpenAI0.1472025-04-16evidence
o4-miniOpenAI0.1472025-04-16evidence
Qwen3 VL 235B A22B ThinkingQwen0.1362025-09-22evidence
Qwen3-235B-A22B-Thinking-2507Qwen0.1822025-07-25evidence
Qwen3.5-122B-A10BQwen0.4752026-02-24evidence
Qwen3.5-27BQwen0.4852026-02-24evidence
Qwen3.5-35B-A3BQwen0.4742026-02-24evidence
Qwen3.5-397B-A17BQwen0.2872026-02-16evidence
Qwen3.6 PlusQwen0.2882026-04-02evidence
Qwen3.6-27BQwen0.242026-04-21evidence
Qwen3.6-35B-A3BQwen0.2142026-04-16evidence
Qwen3.7 MaxQwen0.4142026-05-19evidence
Qwen3.7-PlusQwen0.3472026-05-31evidence
Qwen3.8 MaxQwen0.4362026-08-02evidence
Qwen3.8-27BQwen0.3082026-08-14evidence
Sarvam-105BSarvam AI0.1122026-03-06evidence
Seed 2.1 ProByteDance0.5572026-06-24evidence
Seed 2.1 TurboByteDance0.5462026-06-24evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.

Which sources cite Humanity's Last Exam?

Related math, reasoning, vision benchmarks