Benchmark
Winogrande
WinoGrande: An Adversarial Winograd Schema Challenge at Scale. A large-scale dataset of 44,000 pronoun resolution problems designed to test machine…
- Released
- 2019-11-21
- Modality
- text
- Categories
- reasoning, language
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 22
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Command R+ | Cohere | 0.854 | 2024-08-30 | evidence |
| ERNIE 4.5 | Baidu | 0.513 | 2025-06-25 | evidence |
| Gemma 2 27B | 0.837 | 2024-06-27 | evidence | |
| Gemma 2 9B | 0.806 | 2024-06-27 | evidence | |
| Gemma 3n E2B | 0.668 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.668 | 2025-05-20 | evidence | |
| Gemma 3n E4B | 0.717 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.717 | 2025-05-20 | evidence | |
| GPT-4 | OpenAI | 0.875 | 2023-06-13 | evidence |
| Granite 3.3 8B Base | IBM | 0.744 | 2025-04-16 | evidence |
| Hermes 3 70B | Nous Research | 0.8319 | 2024-08-15 | evidence |
| Llama 3.1 Nemotron 70B Instruct | NVIDIA | 0.8453 | 2024-10-01 | evidence |
| MiMo-V2.5-Pro | Xiaomi | 0.856 | 2026-04-27 | evidence |
| Ministral 8B Instruct | Mistral | 0.753 | 2024-10-16 | evidence |
| Mistral NeMo Instruct | Mistral | 0.768 | 2024-07-18 | evidence |
| Phi 4 Mini | Microsoft | 0.67 | 2025-02-01 | evidence |
| Phi-3.5-mini-instruct | Microsoft | 0.685 | 2024-08-23 | evidence |
| Phi-3.5-MoE-instruct | Microsoft | 0.813 | 2024-08-23 | evidence |
| Qwen2 72B Instruct | Qwen | 0.851 | 2024-07-23 | evidence |
| Qwen2.5 32B Instruct | Qwen | 0.82 | 2024-09-19 | evidence |
| Qwen2.5-Coder 32B Instruct | Qwen | 0.808 | 2024-09-19 | evidence |
| Qwen2.5-Coder 7B Instruct | Qwen | 0.729 | 2024-09-19 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.