Benchmark
AI2D Benchmark
What is AI2D?
AI2D is a dataset of 4,903 illustrative diagrams from grade school natural sciences (such as food webs, human physiology, and life cycles) with over 15,000…
- Released
- 2016-03-24
- Modality
- multimodal
- Evaluates
- multimodal, reasoning, vision
- Openness
- unknown
- Importer
- LLM Stats
- Reported scores
- 33
Source provenance
- Original evidence https://api.zeroeval.com/leaderboard/benchmarks/ai2d?top_n=500
AI2D results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 3.5 Sonnet | Anthropic | 0.947 | 2024-10-22 | evidence |
| DeepSeek VL2 | DeepSeek | 0.814 | 2024-12-13 | evidence |
| DeepSeek VL2 Small | DeepSeek | 0.8 | 2024-12-13 | evidence |
| DeepSeek VL2 Tiny | DeepSeek | 0.716 | 2024-12-13 | evidence |
| Gemma 3 12B | 0.842 | 2025-03-12 | evidence | |
| Gemma 3 27B | 0.845 | 2025-03-12 | evidence | |
| Gemma 3 4B | 0.748 | 2025-03-12 | evidence | |
| GPT-4o | OpenAI | 0.942 | 2024-08-06 | evidence |
| Grok-1.5V | xAI | 0.883 | 2024-04-12 | evidence |
| Llama 3.2 11B Instruct | Meta | 0.911 | 2024-09-25 | evidence |
| Llama 3.2 90B Instruct | Meta | 0.923 | 2024-09-25 | evidence |
| Mistral Small 3.2 24B Instruct | Mistral | 0.9291 | 2025-06-20 | evidence |
| North Micro Vision Instruct | Cohere | 0.775 | 2026-08-12 | evidence |
| Phi-3.5-vision-instruct | Microsoft | 0.781 | 2024-08-23 | evidence |
| Phi-4-multimodal-instruct | Microsoft | 0.823 | 2025-02-01 | evidence |
| Pixtral Large | Mistral | 0.938 | 2024-11-18 | evidence |
| Qwen2.5 VL 72B Instruct | Qwen | 0.884 | 2025-01-26 | evidence |
| Qwen2.5-Omni-7B | Qwen | 0.832 | 2025-03-27 | evidence |
| Qwen3 VL 235B A22B Instruct | Qwen | 0.897 | 2025-09-22 | evidence |
| Qwen3 VL 235B A22B Thinking | Qwen | 0.892 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Instruct | Qwen | 0.85 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Thinking | Qwen | 0.869 | 2025-09-22 | evidence |
| Qwen3 VL 32B Instruct | Qwen | 0.895 | 2025-09-22 | evidence |
| Qwen3 VL 32B Thinking | Qwen | 0.889 | 2025-09-22 | evidence |
| Qwen3 VL 4B Instruct | Qwen | 0.841 | 2025-09-22 | evidence |
| Qwen3 VL 4B Thinking | Qwen | 0.849 | 2025-09-22 | evidence |
| Qwen3 VL 8B Instruct | Qwen | 0.857 | 2025-09-22 | evidence |
| Qwen3 VL 8B Thinking | Qwen | 0.849 | 2025-09-22 | evidence |
| Qwen3.5-122B-A10B | Qwen | 0.933 | 2026-02-24 | evidence |
| Qwen3.5-27B | Qwen | 0.929 | 2026-02-24 | evidence |
| Qwen3.5-35B-A3B | Qwen | 0.926 | 2026-02-24 | evidence |
| Qwen3.6 Plus | Qwen | 0.944 | 2026-04-02 | evidence |
| Qwen3.6-35B-A3B | Qwen | 0.927 | 2026-04-16 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.