Benchmark
MMLU-Redux
An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more…
- Modality
- text
- Categories
- math, reasoning, language, general
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 48
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| DeepSeek-R1-0528 | DeepSeek | 0.934 | 2025-05-28 | evidence |
| DeepSeek-V3 | DeepSeek | 0.891 | 2024-12-25 | evidence |
| DeepSeek-V3.1 | DeepSeek | 0.918 | 2025-01-10 | evidence |
| ERNIE 4.5 | Baidu | 0.432 | 2025-06-25 | evidence |
| Kimi K2 Instruct | Moonshot AI | 0.927 | 2025-07-11 | evidence |
| Kimi K2-Instruct-0905 | Moonshot AI | 0.927 | 2025-09-05 | evidence |
| Kimi K2-Thinking-0905 | Moonshot AI | 0.944 | 2025-09-05 | evidence |
| LongCat-Flash-Thinking | Meituan | 0.893 | 2025-09-22 | evidence |
| MiMo-V2.5-Pro | Xiaomi | 0.928 | 2026-04-27 | evidence |
| Ministral 3 (14B Base 2512) | Mistral | 0.82 | 2025-12-04 | evidence |
| Ministral 3 (3B Base 2512) | Mistral | 0.735 | 2025-12-04 | evidence |
| Ministral 3 (8B Base 2512) | Mistral | 0.793 | 2025-12-04 | evidence |
| Mistral Large 3 | Mistral | 0.82 | 2025-09-01 | evidence |
| Qwen2.5 14B Instruct | Qwen | 0.8 | 2024-09-19 | evidence |
| Qwen2.5 32B Instruct | Qwen | 0.839 | 2024-09-19 | evidence |
| Qwen2.5 72B Instruct | Qwen | 0.868 | 2024-09-19 | evidence |
| Qwen2.5 7B Instruct | Qwen | 0.754 | 2024-09-19 | evidence |
| Qwen2.5-Coder 32B Instruct | Qwen | 0.775 | 2024-09-19 | evidence |
| Qwen2.5-Coder 7B Instruct | Qwen | 0.666 | 2024-09-19 | evidence |
| Qwen2.5-Omni-7B | Qwen | 0.71 | 2025-03-27 | evidence |
| Qwen3 235B A22B | Qwen | 0.874 | 2025-04-29 | evidence |
| Qwen3 VL 235B A22B Instruct | Qwen | 0.922 | 2025-09-22 | evidence |
| Qwen3 VL 235B A22B Thinking | Qwen | 0.937 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Instruct | Qwen | 0.884 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Thinking | Qwen | 0.909 | 2025-09-22 | evidence |
| Qwen3 VL 32B Instruct | Qwen | 0.898 | 2025-09-22 | evidence |
| Qwen3 VL 32B Thinking | Qwen | 0.919 | 2025-09-22 | evidence |
| Qwen3 VL 4B Instruct | Qwen | 0.815 | 2025-09-22 | evidence |
| Qwen3 VL 4B Thinking | Qwen | 0.86 | 2025-09-22 | evidence |
| Qwen3 VL 8B Instruct | Qwen | 0.849 | 2025-09-22 | evidence |
| Qwen3 VL 8B Thinking | Qwen | 0.888 | 2025-09-22 | evidence |
| Qwen3-235B-A22B-Instruct-2507 | Qwen | 0.931 | 2025-07-22 | evidence |
| Qwen3-235B-A22B-Thinking-2507 | Qwen | 0.938 | 2025-07-25 | evidence |
| Qwen3-Next-80B-A3B-Instruct | Qwen | 0.909 | 2025-09-10 | evidence |
| Qwen3-Next-80B-A3B-Thinking | Qwen | 0.925 | 2025-09-10 | evidence |
| Qwen3.5-0.8B | Qwen | 0.595 | 2026-03-02 | evidence |
| Qwen3.5-122B-A10B | Qwen | 0.94 | 2026-02-24 | evidence |
| Qwen3.5-27B | Qwen | 0.932 | 2026-02-24 | evidence |
| Qwen3.5-2B | Qwen | 0.796 | 2026-03-02 | evidence |
| Qwen3.5-35B-A3B | Qwen | 0.933 | 2026-02-24 | evidence |
| Qwen3.5-397B-A17B | Qwen | 0.949 | 2026-02-16 | evidence |
| Qwen3.5-4B | Qwen | 0.888 | 2026-03-02 | evidence |
| Qwen3.5-9B | Qwen | 0.911 | 2026-03-02 | evidence |
| Qwen3.6 Plus | Qwen | 0.945 | 2026-04-02 | evidence |
| Qwen3.6-27B | Qwen | 0.935 | 2026-04-21 | evidence |
| Qwen3.6-35B-A3B | Qwen | 0.933 | 2026-04-16 | evidence |
| Qwen3.7 Max | Qwen | 0.95 | 2026-05-19 | evidence |
| Qwen3.7-Plus | Qwen | 0.945 | 2026-05-31 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.