Benchmark
MATH-500 Benchmark
What is MATH-500?
Largely saturated at the frontier. Benchmark Radar tracks 11 reported scores for MATH-500, cited by 11 source documents, with links to its code.
- Released
- 2021-03-05
- Evaluates
- math
- Openness
- unknown
- Importer
- Model reports
- Reported scores
- 11
Source provenance
- Original evidence https://github.com/openai/prm800k
MATH-500 code
MATH-500 results and reported scores
Model reports
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Bespoke-Stratos-7B | Bespoke Labs | 82.0 | 2025-04-30 | evidence |
| DeepSeek-R1 | DeepSeek | 97.3 | 2025-01-22 | evidence |
| DeepSeek-R1-Distill-Llama-8B | DeepSeek | 86.9 | 2025-04-30 | evidence |
| DeepSeek-R1-Distill-Qwen-7B | DeepSeek | 91.4 | 2025-04-30 | evidence |
| DeepSeek-V3 | DeepSeek | 90.2 | 2024-12-27 | evidence |
| Llama-3.2-3B-Instruct | Meta | 44.4 | 2025-04-30 | evidence |
| o1-mini | OpenAI | 90.0 | 2025-04-30 | evidence |
| OpenThinker-7B | OpenThoughts | 83.0 | 2025-04-30 | evidence |
| Phi-4-Mini (base model, 3.8B) | Microsoft | 71.8 | 2025-04-30 | evidence |
| Phi-4-mini-reasoning (3.8B) | Microsoft | 94.6 | 2025-04-30 | evidence |
| Qwen3-235B-A22B (Thinking) | Qwen | 98.0 | 2025-05-14 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.
Which sources cite MATH-500?
- DeepSeek-V4-Pro DeepSeek, 2026-04-22
- Qwen3 Qwen, 2025-05-14
- Mistral Medium 3 Mistral, 2025-05-07
- Phi-4-mini-reasoning (3.8B) Microsoft, 2025-04-30
- Claude 3.7 Sonnet Anthropic, 2025-02-24
- DeepSeek-R1 DeepSeek, 2025-01-22
- DeepSeek-V3 DeepSeek, 2024-12-27
- Qwen2.5-Coder Qwen, 2024-09-18
- Mistral Large 2 Mistral, 2024-07-24
- Llama 3.1 405B Meta, 2024-07-23
- Gemini 1.5 Google, 2024-03-08