Benchmark
HiddenMath Benchmark
What is HiddenMath?
Google DeepMind's internal mathematical reasoning benchmark that introduces novel problems not encountered during model training to evaluate true mathematical…
- Modality
- text
- Evaluates
- math, reasoning
- Openness
- unknown
- Importer
- LLM Stats
- Reported scores
- 13
Source provenance
- Original evidence https://api.zeroeval.com/leaderboard/benchmarks/hiddenmath?top_n=500
HiddenMath results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Gemini 1.5 Flash | 0.472 | 2024-05-01 | evidence | |
| Gemini 1.5 Flash 8B | 0.328 | 2024-03-15 | evidence | |
| Gemini 1.5 Pro | 0.52 | 2024-05-01 | evidence | |
| Gemini 2.0 Flash | 0.63 | 2024-12-01 | evidence | |
| Gemini 2.0 Flash-Lite | 0.553 | 2025-02-05 | evidence | |
| Gemma 3 12B | 0.545 | 2025-03-12 | evidence | |
| Gemma 3 1B | 0.158 | 2025-03-12 | evidence | |
| Gemma 3 27B | 0.603 | 2025-03-12 | evidence | |
| Gemma 3 4B | 0.43 | 2025-03-12 | evidence | |
| Gemma 3n E2B Instructed | 0.277 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.277 | 2025-05-20 | evidence | |
| Gemma 3n E4B Instructed | 0.377 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.377 | 2025-05-20 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.