Benchmark
MATH
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem…
- Released
- 2021-11-08
- Modality
- text
- Categories
- math, reasoning
- Openness
- open
- Source
- llm_stats
- Reported scores
- 71
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 3 Haiku | Anthropic | 0.389 | 2024-03-13 | evidence |
| Claude 3 Opus | Anthropic | 0.601 | 2024-02-29 | evidence |
| Claude 3 Sonnet | Anthropic | 0.431 | 2024-02-29 | evidence |
| Claude 3.5 Haiku | Anthropic | 0.694 | 2024-10-22 | evidence |
| Claude 3.5 Sonnet | Anthropic | 0.711 | 2024-06-21 | evidence |
| Claude 3.5 Sonnet | Anthropic | 0.783 | 2024-10-22 | evidence |
| DeepSeek-V2.5 | DeepSeek | 0.747 | 2024-05-08 | evidence |
| ERNIE 4.5 | Baidu | 0.124 | 2025-06-25 | evidence |
| Gemini 1.0 Pro | 0.326 | 2024-02-15 | evidence | |
| Gemini 1.5 Flash | 0.779 | 2024-05-01 | evidence | |
| Gemini 1.5 Flash 8B | 0.587 | 2024-03-15 | evidence | |
| Gemini 1.5 Pro | 0.865 | 2024-05-01 | evidence | |
| Gemini 2.0 Flash | 0.897 | 2024-12-01 | evidence | |
| Gemini 2.0 Flash-Lite | 0.868 | 2025-02-05 | evidence | |
| Gemma 2 27B | 0.423 | 2024-06-27 | evidence | |
| Gemma 2 9B | 0.366 | 2024-06-27 | evidence | |
| Gemma 3 12B | 0.838 | 2025-03-12 | evidence | |
| Gemma 3 1B | 0.48 | 2025-03-12 | evidence | |
| Gemma 3 27B | 0.89 | 2025-03-12 | evidence | |
| Gemma 3 4B | 0.756 | 2025-03-12 | evidence | |
| GPT-3.5 Turbo | OpenAI | 0.431 | 2023-03-21 | evidence |
| GPT-4 | OpenAI | 0.42 | 2023-06-13 | evidence |
| GPT-4 Turbo | OpenAI | 0.726 | 2024-04-09 | evidence |
| GPT-4o | OpenAI | 0.766 | 2024-05-13 | evidence |
| GPT-4o mini | OpenAI | 0.702 | 2024-07-18 | evidence |
| GPT-5 | OpenAI | 0.847 | 2025-08-07 | evidence |
| Grok-1.5 | xAI | 0.506 | 2024-03-28 | evidence |
| Grok-2 | xAI | 0.761 | 2024-08-13 | evidence |
| Grok-2 mini | xAI | 0.73 | 2024-08-13 | evidence |
| Hermes 3 70B | Nous Research | 0.208 | 2024-08-15 | evidence |
| Kimi K2 0905 | Moonshot AI | 0.891 | 2025-09-05 | evidence |
| Kimi K2 Base | Moonshot AI | 0.702 | 2025-07-11 | evidence |
| Llama 3.1 405B Instruct | Meta | 0.738 | 2024-07-23 | evidence |
| Llama 3.2 11B Instruct | Meta | 0.519 | 2024-09-25 | evidence |
| Llama 3.2 3B Instruct | Meta | 0.48 | 2024-09-25 | evidence |
| Llama 3.2 90B Instruct | Meta | 0.68 | 2024-09-25 | evidence |
| Llama 3.3 70B Instruct | Meta | 0.77 | 2024-12-06 | evidence |
| Llama 4 Maverick | Meta | 0.612 | 2025-04-05 | evidence |
| Llama 4 Scout | Meta | 0.503 | 2025-04-05 | evidence |
| MiMo-V2.5-Pro | Xiaomi | 0.862 | 2026-04-27 | evidence |
| MiniStral 3 (14B Instruct 2512) | Mistral | 0.904 | 2025-12-04 | evidence |
| Ministral 3 (3B Instruct 2512) | Mistral | 0.83 | 2025-12-04 | evidence |
| Ministral 3 (8B Instruct 2512) | Mistral | 0.876 | 2025-12-04 | evidence |
| Ministral 8B Instruct | Mistral | 0.545 | 2024-10-16 | evidence |
| Mistral Large 3 | Mistral | 0.904 | 2025-09-01 | evidence |
| Mistral Small 3 24B Base | Mistral | 0.4598 | 2025-01-30 | evidence |
| Mistral Small 3 24B Instruct | Mistral | 0.706 | 2025-01-30 | evidence |
| Mistral Small 3.1 24B Instruct | Mistral | 0.693 | 2025-03-17 | evidence |
| Mistral Small 3.2 24B Instruct | Mistral | 0.6942 | 2025-06-20 | evidence |
| Nova Lite | Amazon | 0.733 | 2024-11-20 | evidence |
| Nova Micro | Amazon | 0.693 | 2024-11-20 | evidence |
| Nova Pro | Amazon | 0.766 | 2024-11-20 | evidence |
| o1 | OpenAI | 0.964 | 2024-12-17 | evidence |
| o1-preview | OpenAI | 0.855 | 2024-09-12 | evidence |
| o3-mini | OpenAI | 0.979 | 2025-01-30 | evidence |
| Phi 4 | Microsoft | 0.804 | 2024-12-12 | evidence |
| Phi 4 Mini | Microsoft | 0.64 | 2025-02-01 | evidence |
| Phi-3.5-mini-instruct | Microsoft | 0.485 | 2024-08-23 | evidence |
| Phi-3.5-MoE-instruct | Microsoft | 0.595 | 2024-08-23 | evidence |
| Pixtral-12B | Mistral | 0.481 | 2024-09-17 | evidence |
| Qwen2 72B Instruct | Qwen | 0.597 | 2024-07-23 | evidence |
| Qwen2 7B Instruct | Qwen | 0.496 | 2024-07-23 | evidence |
| Qwen2.5 14B Instruct | Qwen | 0.8 | 2024-09-19 | evidence |
| Qwen2.5 32B Instruct | Qwen | 0.831 | 2024-09-19 | evidence |
| Qwen2.5 72B Instruct | Qwen | 0.831 | 2024-09-19 | evidence |
| Qwen2.5 7B Instruct | Qwen | 0.755 | 2024-09-19 | evidence |
| Qwen2.5 VL 32B Instruct | Qwen | 0.822 | 2025-02-28 | evidence |
| Qwen2.5-Coder 32B Instruct | Qwen | 0.572 | 2024-09-19 | evidence |
| Qwen2.5-Coder 7B Instruct | Qwen | 0.466 | 2024-09-19 | evidence |
| Qwen2.5-Omni-7B | Qwen | 0.715 | 2025-03-27 | evidence |
| Qwen3 235B A22B | Qwen | 0.7184 | 2025-04-29 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.