Benchmark
MGSM
MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the…
- Modality
- text
- Categories
- math, reasoning
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 31
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 3 Haiku | Anthropic | 0.751 | 2024-03-13 | evidence |
| Claude 3 Opus | Anthropic | 0.907 | 2024-02-29 | evidence |
| Claude 3 Sonnet | Anthropic | 0.835 | 2024-02-29 | evidence |
| Claude 3.5 Haiku | Anthropic | 0.856 | 2024-10-22 | evidence |
| Claude 3.5 Sonnet | Anthropic | 0.916 | 2024-06-21 | evidence |
| Claude 3.5 Sonnet | Anthropic | 0.916 | 2024-10-22 | evidence |
| Gemini 1.5 Flash | 0.826 | 2024-05-01 | evidence | |
| Gemini 1.5 Pro | 0.875 | 2024-05-01 | evidence | |
| Gemma 3n E2B Instructed | 0.531 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.531 | 2025-05-20 | evidence | |
| Gemma 3n E4B Instructed | 0.67 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.607 | 2025-05-20 | evidence | |
| GPT-3.5 Turbo | OpenAI | 0.563 | 2023-03-21 | evidence |
| GPT-4 | OpenAI | 0.745 | 2023-06-13 | evidence |
| GPT-4 Turbo | OpenAI | 0.885 | 2024-04-09 | evidence |
| GPT-4o | OpenAI | 0.905 | 2024-05-13 | evidence |
| GPT-4o mini | OpenAI | 0.87 | 2024-07-18 | evidence |
| Llama 3.2 11B Instruct | Meta | 0.689 | 2024-09-25 | evidence |
| Llama 3.2 3B Instruct | Meta | 0.582 | 2024-09-25 | evidence |
| Llama 3.2 90B Instruct | Meta | 0.869 | 2024-09-25 | evidence |
| Llama 3.3 70B Instruct | Meta | 0.911 | 2024-12-06 | evidence |
| Llama 4 Maverick | Meta | 0.923 | 2025-04-05 | evidence |
| Llama 4 Scout | Meta | 0.906 | 2025-04-05 | evidence |
| o1 | OpenAI | 0.893 | 2024-12-17 | evidence |
| o1-preview | OpenAI | 0.908 | 2024-09-12 | evidence |
| o3-mini | OpenAI | 0.92 | 2025-01-30 | evidence |
| Phi 4 | Microsoft | 0.806 | 2024-12-12 | evidence |
| Phi 4 Mini | Microsoft | 0.639 | 2025-02-01 | evidence |
| Phi-3.5-mini-instruct | Microsoft | 0.479 | 2024-08-23 | evidence |
| Phi-3.5-MoE-instruct | Microsoft | 0.587 | 2024-08-23 | evidence |
| Qwen3 235B A22B | Qwen | 0.8353 | 2025-04-29 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.