Benchmark
MRCR v2 (8-needle)
MRCR v2 (8-needle) is a variant of the Multi-Round Coreference Resolution benchmark that includes 8 needle items to retrieve from long contexts. This…
- Modality
- text
- Categories
- long_context, reasoning, general
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 23
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude Opus 4.6 | Anthropic | 0.76 | 2026-02-05 | evidence |
| Gemini 2.5 Pro Preview 06-05 | 0.164 | 2025-06-05 | evidence | |
| Gemini 3 Flash | 0.221 | 2025-12-17 | evidence | |
| Gemini 3 Pro | 0.263 | 2025-11-18 | evidence | |
| Gemini 3.1 Flash-Lite | 0.601 | 2026-03-03 | evidence | |
| Gemini 3.1 Pro | 0.263 | 2026-02-19 | evidence | |
| Gemini 3.5 Flash | 0.266 | 2026-05-19 | evidence | |
| Gemini 3.5 Flash-Lite | 0.213 | 2026-07-21 | evidence | |
| Gemini 3.6 Flash | 0.54 | 2026-07-21 | evidence | |
| Gemini 3.7 Flash | 0.97 | 2026-08-13 | evidence | |
| Gemma 3 27B | 0.135 | 2025-03-12 | evidence | |
| Gemma 4 12B | 0.434 | 2026-05-23 | evidence | |
| Gemma 4 26B-A4B | 0.441 | 2026-04-02 | evidence | |
| Gemma 4 31B | 0.664 | 2026-04-02 | evidence | |
| Gemma 4 E2B | 0.191 | 2026-04-02 | evidence | |
| Gemma 4 E4B | 0.254 | 2026-04-02 | evidence | |
| GPT-5.4 mini | OpenAI | 0.336 | 2026-03-17 | evidence |
| GPT-5.4 nano | OpenAI | 0.331 | 2026-03-17 | evidence |
| GPT-5.5 | OpenAI | 0.74 | 2026-04-23 | evidence |
| GPT-5.6 Luna | OpenAI | 0.413 | 2026-07-09 | evidence |
| GPT-5.6 Sol | OpenAI | 0.915 | 2026-07-09 | evidence |
| GPT-5.6 Terra | OpenAI | 0.896 | 2026-07-09 | evidence |
| Qwen3.8 Max | Qwen | 0.929 | 2026-08-02 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.