Benchmark
Toolathlon Benchmark
What is Toolathlon?
Tool Decathlon is a comprehensive benchmark for evaluating AI agents' ability to use multiple tools across diverse task categories. It measures proficiency in…
- Released
- 2025-10-29
- Modality
- text
- Evaluates
- reasoning, agents, tool calling
- Openness
- open
- Importer
- LLM Stats
- Reported scores
- 37
Source provenance
- Original evidence https://api.zeroeval.com/leaderboard/benchmarks/toolathlon?top_n=500
Toolathlon paper, code and dataset
Toolathlon results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude Opus 4.8 | Anthropic | 0.599 | 2026-05-28 | evidence |
| Claude Sonnet 5 | Anthropic | 0.543 | 2026-06-30 | evidence |
| DeepSeek-V3.2 | DeepSeek | 0.352 | 2025-12-01 | evidence |
| DeepSeek-V3.2 (Thinking) | DeepSeek | 0.352 | 2025-12-01 | evidence |
| DeepSeek-V3.2-Speciale | DeepSeek | 0.352 | 2025-12-01 | evidence |
| DeepSeek-V4-Flash-0423 | DeepSeek | 0.435 | 2026-04-23 | evidence |
| DeepSeek-V4-Flash-0731 | DeepSeek | 0.703 | 2026-07-31 | evidence |
| DeepSeek-V4-Flash-Max | DeepSeek | 0.478 | 2026-04-23 | evidence |
| DeepSeek-V4-Pro-0813 | DeepSeek | 0.741 | 2026-08-13 | evidence |
| DeepSeek-V4-Pro-Max | DeepSeek | 0.518 | 2026-04-23 | evidence |
| Gemini 3 Flash | 0.494 | 2025-12-17 | evidence | |
| Gemini 3.5 Flash | 0.565 | 2026-05-19 | evidence | |
| GLM-5.1 | Z.ai | 0.407 | 2026-04-07 | evidence |
| GLM-5.2 | Z.ai | 0.482 | 2026-06-16 | evidence |
| GLM-5.3 | Z.ai | 0.73 | 2026-08-14 | evidence |
| GPT-5.2 | OpenAI | 0.463 | 2025-12-11 | evidence |
| GPT-5.4 | OpenAI | 0.546 | 2026-03-05 | evidence |
| GPT-5.4 mini | OpenAI | 0.429 | 2026-03-17 | evidence |
| GPT-5.4 nano | OpenAI | 0.355 | 2026-03-17 | evidence |
| GPT-5.5 | OpenAI | 0.556 | 2026-04-23 | evidence |
| GPT-5.6 Luna | OpenAI | 0.534 | 2026-07-09 | evidence |
| GPT-5.6 Sol | OpenAI | 0.58 | 2026-07-09 | evidence |
| GPT-5.6 Terra | OpenAI | 0.531 | 2026-07-09 | evidence |
| Hy3 | Tencent | 0.485 | 2026-07-06 | evidence |
| Inkling-Small | Thinking Machines Lab | 0.544 | 2026-07-30 | evidence |
| Kimi K2.6 | Moonshot AI | 0.5 | 2026-04-20 | evidence |
| Kimi K3 | Moonshot AI | 0.732 | 2026-07-16 | evidence |
| Laguna S 2.1 | Poolside | 0.497 | 2026-07-21 | evidence |
| MiniMax M2.1 | MiniMax | 0.435 | 2025-12-23 | evidence |
| MiniMax M2.7 | MiniMax | 0.463 | 2026-03-18 | evidence |
| Muse Spark 1.1 | Meta | 0.756 | 2026-07-09 | evidence |
| Qwen3.5-397B-A17B | Qwen | 0.383 | 2026-02-16 | evidence |
| Qwen3.6 Plus | Qwen | 0.398 | 2026-04-02 | evidence |
| Qwen3.6-35B-A3B | Qwen | 0.269 | 2026-04-16 | evidence |
| Qwen3.8 Max | Qwen | 0.725 | 2026-08-02 | evidence |
| Seed 2.1 Pro | ByteDance | 0.506 | 2026-06-24 | evidence |
| Seed 2.1 Turbo | ByteDance | 0.491 | 2026-06-24 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.