Benchmark
OSWorld Benchmark
What is OSWorld?
OSWorld: The first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive…
- Released
- 2024-04-11
- Modality
- multimodal
- Evaluates
- multimodal, general, agents, vision
- Openness
- unknown
- Importer
- LLM Stats
- Reported scores
- 20
Source provenance
- Original evidence https://api.zeroeval.com/leaderboard/benchmarks/osworld?top_n=500
OSWorld code
OSWorld results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude Haiku 4.5 | Anthropic | 0.507 | 2025-10-15 | evidence |
| Claude Opus 4.5 | Anthropic | 0.663 | 2025-11-24 | evidence |
| Claude Opus 4.6 | Anthropic | 0.727 | 2026-02-05 | evidence |
| Claude Sonnet 4.5 | Anthropic | 0.614 | 2025-09-29 | evidence |
| Claude Sonnet 4.6 | Anthropic | 0.725 | 2026-02-17 | evidence |
| GLM-5V-Turbo | Z.ai | 0.623 | 2026-04-02 | evidence |
| Qwen2.5 VL 32B Instruct | Qwen | 0.0592 | 2025-02-28 | evidence |
| Qwen2.5 VL 72B Instruct | Qwen | 0.0883 | 2025-01-26 | evidence |
| Qwen3 VL 235B A22B Instruct | Qwen | 0.667 | 2025-09-22 | evidence |
| Qwen3 VL 235B A22B Thinking | Qwen | 0.381 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Instruct | Qwen | 0.303 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Thinking | Qwen | 0.306 | 2025-09-22 | evidence |
| Qwen3 VL 32B Instruct | Qwen | 0.326 | 2025-09-22 | evidence |
| Qwen3 VL 32B Thinking | Qwen | 0.41 | 2025-09-22 | evidence |
| Qwen3 VL 4B Instruct | Qwen | 0.262 | 2025-09-22 | evidence |
| Qwen3 VL 4B Thinking | Qwen | 0.314 | 2025-09-22 | evidence |
| Qwen3 VL 8B Instruct | Qwen | 0.339 | 2025-09-22 | evidence |
| Qwen3 VL 8B Thinking | Qwen | 0.339 | 2025-09-22 | evidence |
| Seed 2.1 Pro | ByteDance | 0.788 | 2026-06-24 | evidence |
| Seed 2.1 Turbo | ByteDance | 0.764 | 2026-06-24 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.