Benchmark
ARC-C
The AI2 Reasoning Challenge (ARC) Challenge Set is a multiple-choice question-answering benchmark containing grade-school level science questions that…
- Modality
- text
- Categories
- reasoning, general
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 34
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 3 Haiku | Anthropic | 0.892 | 2024-03-13 | evidence |
| Claude 3 Opus | Anthropic | 0.964 | 2024-02-29 | evidence |
| Claude 3 Sonnet | Anthropic | 0.932 | 2024-02-29 | evidence |
| Command R+ | Cohere | 0.7099 | 2024-08-30 | evidence |
| ERNIE 4.5 | Baidu | 0.406 | 2025-06-25 | evidence |
| Gemma 2 27B | 0.714 | 2024-06-27 | evidence | |
| Gemma 2 9B | 0.684 | 2024-06-27 | evidence | |
| Gemma 3n E2B | 0.517 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.517 | 2025-05-20 | evidence | |
| Gemma 3n E4B | 0.616 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.616 | 2025-05-20 | evidence | |
| Granite 3.3 8B Base | IBM | 0.5084 | 2025-04-16 | evidence |
| Hermes 3 70B | Nous Research | 0.6553 | 2024-08-15 | evidence |
| Jamba 1.5 Large | AI21 Labs | 0.93 | 2024-08-22 | evidence |
| Jamba 1.5 Mini | AI21 Labs | 0.857 | 2024-08-22 | evidence |
| Llama 3.1 405B Instruct | Meta | 0.969 | 2024-07-23 | evidence |
| Llama 3.1 70B Instruct | Meta | 0.948 | 2024-07-23 | evidence |
| Llama 3.1 8B Instruct | Meta | 0.834 | 2024-07-23 | evidence |
| Llama 3.1 Nemotron 70B Instruct | NVIDIA | 0.692 | 2024-10-01 | evidence |
| Llama 3.2 3B Instruct | Meta | 0.786 | 2024-09-25 | evidence |
| MiMo-V2.5-Pro | Xiaomi | 0.972 | 2026-04-27 | evidence |
| Ministral 8B Instruct | Mistral | 0.719 | 2024-10-16 | evidence |
| Mistral Small 3 24B Base | Mistral | 0.9129 | 2025-01-30 | evidence |
| Nova Lite | Amazon | 0.924 | 2024-11-20 | evidence |
| Nova Micro | Amazon | 0.902 | 2024-11-20 | evidence |
| Nova Pro | Amazon | 0.948 | 2024-11-20 | evidence |
| Phi 4 Mini | Microsoft | 0.837 | 2025-02-01 | evidence |
| Phi-3.5-mini-instruct | Microsoft | 0.846 | 2024-08-23 | evidence |
| Phi-3.5-MoE-instruct | Microsoft | 0.91 | 2024-08-23 | evidence |
| Qwen2 72B Instruct | Qwen | 0.689 | 2024-07-23 | evidence |
| Qwen2.5 14B Instruct | Qwen | 0.673 | 2024-09-19 | evidence |
| Qwen2.5 32B Instruct | Qwen | 0.704 | 2024-09-19 | evidence |
| Qwen2.5-Coder 32B Instruct | Qwen | 0.705 | 2024-09-19 | evidence |
| Qwen2.5-Coder 7B Instruct | Qwen | 0.609 | 2024-09-19 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.