ClawBench
Explore score history ↗Agents complete everyday tasks on live production websites. The paper evaluates 153 tasks with trajectory judging against human references; the current repository lists a changed 152-task V1 corpus. V2 uses request interception and separate lenient and strict judge rubrics. Corpus revision, harness and judge change the measurement, and live websites can change between runs.
- Released
- 2026-04-09
- Evaluates
- agent
- Openness
- unknown
- Importer
- Model reports
ClawBench paper
ClawBench results and reported scores
Reported benchmark scores · Model reports
Reported scores: 8
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude Haiku 4.5 | Anthropic | 18.3 | 2026-07-20 | evidence |
| Claude Sonnet 4.6 | Anthropic | 33.3 | 2026-07-20 | evidence |
| Gemini 3 Flash | 19.0 | 2026-07-20 | evidence | |
| Gemini 3.1 Flash-Lite | 3.3 | 2026-07-20 | evidence | |
| Gemini 3.1 Pro | 9.8 | 2026-07-20 | evidence | |
| GLM-5 | Z.ai | 24.2 | 2026-07-20 | evidence |
| GPT-5.4 | OpenAI | 6.5 | 2026-07-20 | evidence |
| Qwen3.5-397B-A17B | Qwen | 26.1 | 2026-07-20 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.
Which sources cite ClawBench?
- ClawBench paper Table 3 (arXiv v2) TIGER-AI-Lab, 2026-07-20