HellaSwag Benchmark
Explore score history ↗10,000 questions is the sample count the study audited (arXiv:2607.28801, Table 1), not the full dataset of 70k problems (arXiv:1905.07830). Sentence-completion benchmark (arXiv:1905.07830, "HellaSwag: Can a Machine Really Finish Your Sentence?"). The study notes concerns persist about validity and data contamination (arXiv:2607.28801, section 2). The annotating paper also reports full-split accuracies for Llama-3.2-1B and SmolLM-1.7B-Instruct (arXiv:2607.28801, Table 4 / App. Table 9); those rows are filed in benchmark_scores.yml.
- Released
- 2019-05-19
- Evaluates
- reasoning
- Openness
- unknown
- Importer
- Model reports
HellaSwag paper
HellaSwag results and reported scores
Reported benchmark scores · Model reports
Reported scores: 2
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.
Which sources cite HellaSwag?
- Llama-3.2-1B and SmolLM-1.7B-Instruct Aleph Alpha Research, 2026-07-30