Benchmark
AIME 2024 Benchmark
What is AIME 2024?
American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each…
- Modality
- text
- Evaluates
- math, reasoning
- Openness
- unknown
- Importer
- LLM Stats
- Reported scores
- 53
Source provenance
- Original evidence https://api.zeroeval.com/leaderboard/benchmarks/aime-2024?top_n=500
AIME 2024 results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 3.7 Sonnet | Anthropic | 0.8 | 2025-02-24 | evidence |
| DeepSeek R1 Distill Llama 70B | DeepSeek | 0.867 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Llama 8B | DeepSeek | 0.8 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Qwen 1.5B | DeepSeek | 0.527 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Qwen 14B | DeepSeek | 0.8 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Qwen 32B | DeepSeek | 0.833 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Qwen 7B | DeepSeek | 0.833 | 2025-01-20 | evidence |
| DeepSeek R1 Zero | DeepSeek | 0.867 | 2025-01-20 | evidence |
| DeepSeek-R1-0528 | DeepSeek | 0.914 | 2025-05-28 | evidence |
| DeepSeek-V3 | DeepSeek | 0.392 | 2024-12-25 | evidence |
| DeepSeek-V3 0324 | DeepSeek | 0.594 | 2025-03-25 | evidence |
| DeepSeek-V3.1 | DeepSeek | 0.663 | 2025-01-10 | evidence |
| Gemini 2.0 Flash Thinking | 0.733 | 2025-01-21 | evidence | |
| Gemini 2.5 Flash | 0.88 | 2025-05-20 | evidence | |
| Gemini 2.5 Pro | 0.92 | 2025-05-20 | evidence | |
| GLM-4.5 | Z.ai | 0.91 | 2025-07-28 | evidence |
| GLM-4.5-Air | Z.ai | 0.894 | 2025-07-28 | evidence |
| GPT-4.1 | OpenAI | 0.481 | 2025-04-14 | evidence |
| GPT-4.1 mini | OpenAI | 0.496 | 2025-04-14 | evidence |
| GPT-4.1 nano | OpenAI | 0.294 | 2025-04-14 | evidence |
| GPT-4.5 | OpenAI | 0.367 | 2025-02-27 | evidence |
| GPT-4o | OpenAI | 0.131 | 2024-08-06 | evidence |
| Granite 3.3 8B Base | IBM | 0.812 | 2025-04-16 | evidence |
| Granite 3.3 8B Instruct | IBM | 0.812 | 2025-04-16 | evidence |
| Grok-3 | xAI | 0.933 | 2025-02-17 | evidence |
| Grok-3 Mini | xAI | 0.958 | 2025-02-17 | evidence |
| Kimi K2 0905 | Moonshot AI | 0.72 | 2025-09-05 | evidence |
| Kimi K2 Instruct | Moonshot AI | 0.696 | 2025-07-11 | evidence |
| Kimi K2-Instruct-0905 | Moonshot AI | 0.696 | 2025-09-05 | evidence |
| Kimi-k1.5 | Moonshot AI | 0.775 | 2025-01-20 | evidence |
| LongCat-Flash-Lite | Meituan | 0.7219 | 2026-02-05 | evidence |
| LongCat-Flash-Thinking | Meituan | 0.933 | 2025-09-22 | evidence |
| Magistral Medium | Mistral | 0.736 | 2025-06-10 | evidence |
| Magistral Small 2506 | Mistral | 0.7068 | 2025-06-10 | evidence |
| Min istral 3 (3B Reasoning 2512) | Mistral | 0.775 | 2025-12-04 | evidence |
| MiniCPM-SALA | OpenBMB | 0.8375 | 2026-02-11 | evidence |
| MiniMax M1 40K | MiniMax | 0.833 | 2025-06-16 | evidence |
| MiniMax M1 80K | MiniMax | 0.86 | 2025-06-16 | evidence |
| Ministral 3 (14B Reasoning 2512) | Mistral | 0.898 | 2025-12-04 | evidence |
| Ministral 3 (8B Reasoning 2512) | Mistral | 0.86 | 2025-12-04 | evidence |
| o1 | OpenAI | 0.743 | 2024-12-17 | evidence |
| o1-preview | OpenAI | 0.42 | 2024-09-12 | evidence |
| o1-pro | OpenAI | 0.86 | 2024-12-17 | evidence |
| o3 | OpenAI | 0.916 | 2025-04-16 | evidence |
| o3-mini | OpenAI | 0.873 | 2025-01-30 | evidence |
| o4-mini | OpenAI | 0.934 | 2025-04-16 | evidence |
| Phi 4 Reasoning | Microsoft | 0.753 | 2025-04-30 | evidence |
| Phi 4 Reasoning Plus | Microsoft | 0.813 | 2025-04-30 | evidence |
| Qwen3 235B A22B | Qwen | 0.857 | 2025-04-29 | evidence |
| Qwen3 30B A3B | Qwen | 0.804 | 2025-04-29 | evidence |
| Qwen3 32B | Qwen | 0.814 | 2025-04-29 | evidence |
| QwQ-32B | Qwen | 0.795 | 2025-03-05 | evidence |
| QwQ-32B-Preview | Qwen | 0.5 | 2024-11-28 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.