Benchmark
Natural Questions Benchmark
What is Natural Questions?
Natural Questions is a question answering dataset featuring real anonymized queries issued to Google search engine. It contains 307,373 training examples where…
- Modality
- text
- Evaluates
- reasoning, search, general
- Openness
- unknown
- Importer
- LLM Stats
- Reported scores
- 7
Source provenance
Natural Questions results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Gemma 2 27B | 0.345 | 2024-06-27 | evidence | |
| Gemma 2 9B | 0.292 | 2024-06-27 | evidence | |
| Gemma 3n E2B | 0.155 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.155 | 2025-05-20 | evidence | |
| Gemma 3n E4B | 0.209 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.209 | 2025-05-20 | evidence | |
| Mistral NeMo Instruct | Mistral | 0.312 | 2024-07-18 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.