Benchmark
DROP
DROP (Discrete Reasoning Over Paragraphs) is a reading comprehension benchmark requiring discrete reasoning over paragraph content. It contains…
- Modality
- text
- Categories
- math, reasoning
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 30
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 3 Haiku | Anthropic | 0.784 | 2024-03-13 | evidence |
| Claude 3 Opus | Anthropic | 0.831 | 2024-02-29 | evidence |
| Claude 3 Sonnet | Anthropic | 0.789 | 2024-02-29 | evidence |
| Claude 3.5 Haiku | Anthropic | 0.831 | 2024-10-22 | evidence |
| Claude 3.5 Sonnet | Anthropic | 0.871 | 2024-06-21 | evidence |
| Claude 3.5 Sonnet | Anthropic | 0.871 | 2024-10-22 | evidence |
| DeepSeek-V3 | DeepSeek | 0.916 | 2024-12-25 | evidence |
| ERNIE 4.5 | Baidu | 0.286 | 2025-06-25 | evidence |
| Gemini 1.5 Pro | 0.749 | 2024-05-01 | evidence | |
| Gemma 3n E2B | 0.539 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.539 | 2025-05-20 | evidence | |
| Gemma 3n E4B | 0.608 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.608 | 2025-05-20 | evidence | |
| GPT-3.5 Turbo | OpenAI | 0.702 | 2023-03-21 | evidence |
| GPT-4 | OpenAI | 0.809 | 2023-06-13 | evidence |
| GPT-4 Turbo | OpenAI | 0.86 | 2024-04-09 | evidence |
| GPT-4o | OpenAI | 0.834 | 2024-05-13 | evidence |
| GPT-4o mini | OpenAI | 0.797 | 2024-07-18 | evidence |
| Granite 3.3 8B Base | IBM | 0.3614 | 2025-04-16 | evidence |
| Granite 3.3 8B Instruct | IBM | 0.5936 | 2025-04-16 | evidence |
| IBM Granite 4.0 Tiny Preview | IBM | 0.462 | 2025-05-02 | evidence |
| Llama 3.1 405B Instruct | Meta | 0.848 | 2024-07-23 | evidence |
| Llama 3.1 70B Instruct | Meta | 0.796 | 2024-07-23 | evidence |
| Llama 3.1 8B Instruct | Meta | 0.595 | 2024-07-23 | evidence |
| LongCat-Flash-Chat | Meituan | 0.7906 | 2025-08-29 | evidence |
| MiMo-V2.5-Pro | Xiaomi | 0.863 | 2026-04-27 | evidence |
| Nova Lite | Amazon | 0.802 | 2024-11-20 | evidence |
| Nova Micro | Amazon | 0.793 | 2024-11-20 | evidence |
| Nova Pro | Amazon | 0.854 | 2024-11-20 | evidence |
| Phi 4 | Microsoft | 0.755 | 2024-12-12 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.