ARC Benchmark
Explore score history ↗3,550 questions is the sample count the study audited (arXiv:2607.28801, Table 1), not the full corpus of 7,787 questions (arXiv:1803.05457). A question-answering benchmark (arXiv:1803.05457, "Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge"). The study notes its small scale and multiple-choice format limit its discriminative power (arXiv:2607.28801, section 2). The annotating paper also reports full-split accuracies for Llama-3.2-1B and SmolLM-1.7B-Instruct (arXiv:2607.28801, Table 4 / App. Table 9); those rows are filed in benchmark_scores.yml.
- Released
- 2018-03-14
- Evaluates
- reasoning
- Openness
- unknown
- Importer
- Model reports
ARC paper
ARC results and reported scores
Reported benchmark scores · Model reports
Reported scores: 2
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.
Which sources cite ARC?
- Llama-3.2-1B and SmolLM-1.7B-Instruct Aleph Alpha Research, 2026-07-30