CharXiv-D Benchmark
Explore score history ↗CharXiv-D is the descriptive questions subset of the CharXiv benchmark, designed to assess multimodal large language models' ability to extract basic information from scientific charts. It contains descriptive questions covering information extraction, enumeration, pattern recognition, and counting across 2,323 diverse charts from arXiv papers, all curated and verified by human experts.
- Modality
- multimodal
- Evaluates
- multimodal, reasoning, structured output, vision
- Openness
- unknown
- Importer
- LLM Stats
Source provenance
CharXiv-D results and reported scores
Reported benchmark scores · LLM Stats
Reported scores: 17
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Command A+ | Cohere | 0.88 | 2026-05-20 | evidence |
| GPT-4.1 | OpenAI | 0.879 | 2025-04-14 | evidence |
| GPT-4.1 mini | OpenAI | 0.884 | 2025-04-14 | evidence |
| GPT-4.1 nano | OpenAI | 0.739 | 2025-04-14 | evidence |
| GPT-4.5 | OpenAI | 0.9 | 2025-02-27 | evidence |
| GPT-4o | OpenAI | 0.853 | 2024-08-06 | evidence |
| North Micro Vision Instruct | Cohere | 0.6 | 2026-08-12 | evidence |
| Qwen3 VL 30B A3B Instruct | Qwen | 0.855 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Thinking | Qwen | 0.869 | 2025-09-22 | evidence |
| Qwen3 VL 32B Instruct | Qwen | 0.905 | 2025-09-22 | evidence |
| Qwen3 VL 32B Thinking | Qwen | 0.902 | 2025-09-22 | evidence |
| Qwen3 VL 4B Instruct | Qwen | 0.762 | 2025-09-22 | evidence |
| Qwen3 VL 4B Thinking | Qwen | 0.839 | 2025-09-22 | evidence |
| Qwen3 VL 8B Instruct | Qwen | 0.83 | 2025-09-22 | evidence |
| Qwen3 VL 8B Thinking | Qwen | 0.859 | 2025-09-22 | evidence |
| Seed 2.1 Pro | ByteDance | 0.955 | 2026-06-24 | evidence |
| Seed 2.1 Turbo | ByteDance | 0.946 | 2026-06-24 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.