Benchmark

MMLU-Redux Benchmark

What is MMLU-Redux?

Error-corrected MMLU subset; reported mainly by open-weight cards. Benchmark Radar tracks 4 reported scores for MMLU-Redux, cited by 5 source documents, with…

Released: 2024-06-06
Evaluates: knowledge
Openness: unknown
Importer: Model reports
Reported scores: 4

Source provenance

Original evidence https://github.com/aryopg/mmlu-redux

MMLU-Redux code

Code https://github.com/aryopg/mmlu-redux

MMLU-Redux results and reported scores

Model reports

Model	Organization	Reported value	Reported	Evidence
DeepSeek-R1	DeepSeek	92.9	2025-01-22	evidence
DeepSeek-V3	DeepSeek	89.1	2024-12-27	evidence
Qwen3-235B-A22B (Thinking)	Qwen	92.7	2025-05-14	evidence
Qwen3.5-397B-A17B	Qwen	94.9	2026-02-16	evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.

Which sources cite MMLU-Redux?

DeepSeek-V4-Pro DeepSeek, 2026-04-22
Qwen3.5-397B-A17B Qwen, 2026-02-16
Qwen3 Qwen, 2025-05-14
DeepSeek-R1 DeepSeek, 2025-01-22
DeepSeek-V3 DeepSeek, 2024-12-27