YapBench
Explore score history ↗Lower is better. YapScore counts characters a response adds beyond a curated minimal-sufficient answer, and YapIndex is the equal-weight mean of the three category medians (arXiv:2601.00624). It measures verbosity on brevity-ideal prompts, not correctness: a shorter answer can be a worse one. The paper evaluates 76 configurations on version 0.1, and the live leaderboard updates separately, so prompt-set version and reasoning mode change the number.
- Released
- 2026-01-02
- Evaluates
- general
- Openness
- unknown
- Importer
- Model reports
YapBench paper
YapBench results and reported scores
Reported benchmark scores · Model reports
Reported scores: 76
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| claude-3.5-haiku | Anthropic | 401.2 | 2026-01-02 | evidence |
| claude-3.5-sonnet | Anthropic | 199.7 | 2026-01-02 | evidence |
| claude-haiku-4.5 | Anthropic | 333.2 | 2026-01-02 | evidence |
| claude-haiku-4.5 (reasoning) | Anthropic | 335.2 | 2026-01-02 | evidence |
| claude-opus-4.5 | Anthropic | 97.0 | 2026-01-02 | evidence |
| claude-opus-4.5 (reasoning) | Anthropic | 99.2 | 2026-01-02 | evidence |
| claude-sonnet-4.5 | Anthropic | 285.0 | 2026-01-02 | evidence |
| claude-sonnet-4.5 (reasoning) | Anthropic | 278.7 | 2026-01-02 | evidence |
| deepseek-chat | DeepSeek | 345.7 | 2026-01-02 | evidence |
| deepseek-v3.2 | DeepSeek | 426.2 | 2026-01-02 | evidence |
| deepseek-v3.2 (reasoning) | DeepSeek | 215.0 | 2026-01-02 | evidence |
| devstral-2512 | Mistral | 449.0 | 2026-01-02 | evidence |
| gemini-2.0-flash-001 | 326.5 | 2026-01-02 | evidence | |
| gemini-2.0-flash-lite-001 | 227.7 | 2026-01-02 | evidence | |
| gemini-2.5-flash | 210.5 | 2026-01-02 | evidence | |
| gemini-2.5-flash (reasoning) | 195.2 | 2026-01-02 | evidence | |
| gemini-2.5-flash-lite | 476.2 | 2026-01-02 | evidence | |
| gemini-2.5-flash-lite (reasoning) | 280.5 | 2026-01-02 | evidence | |
| gemini-2.5-pro | 539.8 | 2026-01-02 | evidence | |
| gemini-3-flash-preview | 339.3 | 2026-01-02 | evidence | |
| gemini-3-flash-preview (reasoning) | 340.8 | 2026-01-02 | evidence | |
| gemini-3-pro-preview | 226.0 | 2026-01-02 | evidence | |
| glm-4.5 | Z.ai | 1427.0 | 2026-01-02 | evidence |
| glm-4.6:exacto | Z.ai | 930.7 | 2026-01-02 | evidence |
| glm-4.6v | Z.ai | 410.5 | 2026-01-02 | evidence |
| glm-4.7 | Z.ai | 66.3 | 2026-01-02 | evidence |
| gpt-3.5-turbo | OpenAI | 22.7 | 2026-01-02 | evidence |
| gpt-4 | OpenAI | 51.2 | 2026-01-02 | evidence |
| gpt-4-turbo | OpenAI | 252.5 | 2026-01-02 | evidence |
| gpt-4.1 | OpenAI | 168.0 | 2026-01-02 | evidence |
| gpt-4.1-mini | OpenAI | 75.7 | 2026-01-02 | evidence |
| gpt-4.1-nano | OpenAI | 105.2 | 2026-01-02 | evidence |
| gpt-4o | OpenAI | 167.2 | 2026-01-02 | evidence |
| gpt-4o-mini | OpenAI | 183.2 | 2026-01-02 | evidence |
| gpt-5 | OpenAI | 100.7 | 2026-01-02 | evidence |
| gpt-5-mini | OpenAI | 250.5 | 2026-01-02 | evidence |
| gpt-5-nano | OpenAI | 175.0 | 2026-01-02 | evidence |
| gpt-5.1 | OpenAI | 111.8 | 2026-01-02 | evidence |
| gpt-5.1 (reasoning) | OpenAI | 120.8 | 2026-01-02 | evidence |
| gpt-5.1-codex | OpenAI | 85.3 | 2026-01-02 | evidence |
| gpt-5.2 | OpenAI | 86.8 | 2026-01-02 | evidence |
| gpt-5.2 (reasoning) | OpenAI | 89.8 | 2026-01-02 | evidence |
| gpt-oss-120b | OpenAI | 269.7 | 2026-01-02 | evidence |
| gpt-oss-20b | OpenAI | 199.7 | 2026-01-02 | evidence |
| grok-4 | xAI | 126.3 | 2026-01-02 | evidence |
| grok-4-fast | xAI | 70.2 | 2026-01-02 | evidence |
| grok-4.1-fast | xAI | 98.0 | 2026-01-02 | evidence |
| grok-code-fast-1 | xAI | 284.5 | 2026-01-02 | evidence |
| kimi-k2-0905 | Moonshot AI | 44.7 | 2026-01-02 | evidence |
| llama-3.1-8b-instruct | Meta | 315.7 | 2026-01-02 | evidence |
| llama-3.3-70b-instruct | Meta | 312.8 | 2026-01-02 | evidence |
| llama-4-maverick | Meta | 342.8 | 2026-01-02 | evidence |
| llama-4-scout | Meta | 350.5 | 2026-01-02 | evidence |
| minimax-m2 | MiniMax | 297.3 | 2026-01-02 | evidence |
| mistral-large-2512 | Mistral | 444.0 | 2026-01-02 | evidence |
| mistral-medium-3 | Mistral | 361.0 | 2026-01-02 | evidence |
| mistral-small-3.1-24b-instruct | Mistral | 389.2 | 2026-01-02 | evidence |
| nova-2-lite-v1 | Amazon | 896.3 | 2026-01-02 | evidence |
| nova-2-lite-v1 (reasoning) | Amazon | 788.3 | 2026-01-02 | evidence |
| o3 | OpenAI | 84.3 | 2026-01-02 | evidence |
| o3-mini | OpenAI | 153.2 | 2026-01-02 | evidence |
| o4-mini | OpenAI | 115.5 | 2026-01-02 | evidence |
| olmo-2-0325-32b-instruct | Allen Institute for AI | 268.7 | 2026-01-02 | evidence |
| olmo-3.1-32b-think:free | Allen Institute for AI | 530.5 | 2026-01-02 | evidence |
| phi-4-reasoning-plus | Microsoft | 281.2 | 2026-01-02 | evidence |
| phi-4-reasoning-plus (reasoning) | Microsoft | 281.0 | 2026-01-02 | evidence |
| qwen-plus | Qwen | 267.3 | 2026-01-02 | evidence |
| qwen-plus (reasoning) | Qwen | 800.3 | 2026-01-02 | evidence |
| qwen3-14b | Qwen | 558.0 | 2026-01-02 | evidence |
| qwen3-235b-a22b-2507 | Qwen | 264.0 | 2026-01-02 | evidence |
| qwen3-235b-a22b-2507 (reasoning) | Qwen | 260.3 | 2026-01-02 | evidence |
| qwen3-32b | Qwen | 725.3 | 2026-01-02 | evidence |
| qwen3-coder | Qwen | 326.3 | 2026-01-02 | evidence |
| qwen3-coder (reasoning) | Qwen | 325.5 | 2026-01-02 | evidence |
| qwen3-vl-235b-a22b-thinking | Qwen | 602.8 | 2026-01-02 | evidence |
| qwen3-vl-8b-instruct | Qwen | 518.0 | 2026-01-02 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.
Which sources cite YapBench?
- YapBench v0.1 full leaderboard (Table 6) YapBench authors, 2026-01-02