Benchmark
HellaSwag
A challenging commonsense natural language inference dataset that uses Adversarial Filtering to create questions trivial for humans (>95% accuracy) but…
- Released
- 2019-05-19
- Modality
- text
- Categories
- reasoning
- Openness
- restricted
- Source
- llm_stats
- Reported scores
- 27
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 3 Haiku | Anthropic | 0.859 | 2024-03-13 | evidence |
| Claude 3 Opus | Anthropic | 0.954 | 2024-02-29 | evidence |
| Claude 3 Sonnet | Anthropic | 0.89 | 2024-02-29 | evidence |
| Command R+ | Cohere | 0.886 | 2024-08-30 | evidence |
| ERNIE 4.5 | Baidu | 0.33 | 2025-06-25 | evidence |
| Gemini 1.5 Flash | 0.865 | 2024-05-01 | evidence | |
| Gemini 1.5 Pro | 0.933 | 2024-05-01 | evidence | |
| Gemma 2 27B | 0.864 | 2024-06-27 | evidence | |
| Gemma 2 9B | 0.819 | 2024-06-27 | evidence | |
| Gemma 3n E2B | 0.722 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.722 | 2025-05-20 | evidence | |
| Gemma 3n E4B | 0.786 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.786 | 2025-05-20 | evidence | |
| GPT-4 | OpenAI | 0.953 | 2023-06-13 | evidence |
| Granite 3.3 8B Base | IBM | 0.801 | 2025-04-16 | evidence |
| Hermes 3 70B | Nous Research | 0.8819 | 2024-08-15 | evidence |
| Llama 3.1 Nemotron 70B Instruct | NVIDIA | 0.8558 | 2024-10-01 | evidence |
| Llama 3.2 3B Instruct | Meta | 0.698 | 2024-09-25 | evidence |
| MiMo-V2.5-Pro | Xiaomi | 0.898 | 2026-04-27 | evidence |
| Mistral NeMo Instruct | Mistral | 0.835 | 2024-07-18 | evidence |
| Phi 4 Mini | Microsoft | 0.691 | 2025-02-01 | evidence |
| Phi-3.5-mini-instruct | Microsoft | 0.694 | 2024-08-23 | evidence |
| Phi-3.5-MoE-instruct | Microsoft | 0.838 | 2024-08-23 | evidence |
| Qwen2 72B Instruct | Qwen | 0.876 | 2024-07-23 | evidence |
| Qwen2.5 32B Instruct | Qwen | 0.852 | 2024-09-19 | evidence |
| Qwen2.5-Coder 32B Instruct | Qwen | 0.83 | 2024-09-19 | evidence |
| Qwen2.5-Coder 7B Instruct | Qwen | 0.768 | 2024-09-19 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.