Benchmark
SWE-Bench Multimodal
SWE-Bench Multimodal extends SWE-Bench to evaluate language models on software engineering tasks that involve visual inputs such as screenshots, UI…
- Modality
- multimodal
- Categories
- multimodal, reasoning, agents, code, vision
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 4
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude Mythos Preview | Anthropic | 0.59 | 2026-04-07 | evidence |
| Claude Opus 4.8 | Anthropic | 0.384 | 2026-05-28 | evidence |
| Claude Sonnet 5 | Anthropic | 0.281 | 2026-06-30 | evidence |
| Qwen3.8-27B | Qwen | 0.386 | 2026-08-14 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.