Benchmark
Global-MMLU-Lite Benchmark
What is Global-MMLU-Lite?
A lightweight version of Global MMLU benchmark that evaluates language models across multiple languages while addressing cultural and linguistic biases in…
- Modality
- text
- Evaluates
- reasoning, language, general
- Openness
- unknown
- Importer
- LLM Stats
- Reported scores
- 15
Source provenance
Global-MMLU-Lite results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Gemini 2.0 Flash-Lite | 0.782 | 2025-02-05 | evidence | |
| Gemini 2.5 Flash | 0.884 | 2025-05-20 | evidence | |
| Gemini 2.5 Flash-Lite | 0.811 | 2025-06-17 | evidence | |
| Gemini 2.5 Pro | 0.886 | 2025-05-20 | evidence | |
| Gemini 2.5 Pro Preview 06-05 | 0.892 | 2025-06-05 | evidence | |
| Gemini Diffusion | 0.691 | 2025-05-20 | evidence | |
| Gemma 3 12B | 0.695 | 2025-03-12 | evidence | |
| Gemma 3 1B | 0.342 | 2025-03-12 | evidence | |
| Gemma 3 27B | 0.751 | 2025-03-12 | evidence | |
| Gemma 3 4B | 0.545 | 2025-03-12 | evidence | |
| Gemma 3n E2B Instructed | 0.59 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.59 | 2025-05-20 | evidence | |
| Gemma 3n E4B Instructed | 0.645 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.645 | 2025-05-20 | evidence | |
| Inkling-Small | Thinking Machines Lab | 0.867 | 2026-07-30 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.